Collective decision making, leadership, and collective intelligence: Tests with agent-based simulations and a Field study

a b s t r a c t

This multi-level (individual and collective) study examines collective decision making as it
relates to the performance metric of collective decision quality. A collectivistic leadership
approach is used, as leaderless collectives engaged in decision making are inherently involved
in collective leadership. A multi-level conceptual model for collective decision making is introduced,
which incorporates leadership and collective intelligence. Using agent-based simulations
and content-coded field study data, results from both methods suggest that there is a positive
relationship between individual and collective intelligence, as well as a positive relationship
between collective intelligence and collective decision quality. The implications of these and
related findings for future collective level research bridging the fields of decision making, leadership,
and collective intelligence are discussed.

Within the field of organizational behavior, the areas of leadership and decision making are among the most highly
studied topics, with entire books (Bass, 2008; Guzzo & Salas, 1995; Keeney & Raiffa, 1976; Yukl, 2009) and even journals
(e.g., Leadership Quarterly, Organizational Behavior and Human Decision Processes) dedicated solely to this research. These areas
have a long history of being studied together (Vroom & Yetton, 1973), with research often examining a singular leader in conjunction
with a group which he or she leads. The inherent nature of researching leader-led groups requires the careful consideration
of multiple levels of analysis, as there are individuals nested within groups being led by another individual and the levels of
individual, dyad, and group exist simultaneously. Levels of analysis were first introduced into organizational behavior research
almost thirty years ago (Dansereau, Alutto, & Yammarino, 1984; Rousseau, 1985) and there have been many pleas for greater attention
to levels in organizational behavior in general (Dansereau, Yammarino, & Kohles, 1999; House, Rousseau, & Thomas-Hunt,
1995; Klein, Dansereau, & Hall, 1994) and leadership in particular (e.g.Dansereau & Yammarino, 1998; Dionne et al., 2014;
Schriesheim, Castro, Zhou, & Yammarino, 2001; Yammarino & Dansereau, 2009; Yammarino, Dionne, Chun, & Dansereau, 2005;
Dionne, Chun, Hao, Serban, Yammarino, & Spangler, 2012) since then. Despite increasing awareness and study of levels of analysis,
the discussion of leadership and decision making does not typically extend to the collective level of analysis.

In spite of a lack of research, many businesses have discovered collective decision making as a valuable tool they can use to
solve large, complex problems which would be too time consuming for an individual, or even a group, to attempt. One such
organization is Google, whose goal is to rank places to find information on the internet. When an internet user adds a link
from their own webpage to another, they are endorsing it as being important. The more people that link to a page, the more
important the Google will determine it is (Langville & Meyer, 2003). When a search is performed, webpages determined to be
most important are listed first. Thus, Google utilizes the knowledge of internet users to quickly determine the best place to
find information, a complex problem. Challenges facing organizations are continually becoming more complex and teams have
emerged as a way to counteract that complexity. If this trend continues, increasing complexity will eventually overwhelm the
abilities of teams, forcing organizations to rely more on collectives and their decision making capabilities.
In the study of collective decision making there are at least two key distinctions: first, the level of analysis is the collective, and
second, the argument can be made that leaderless collectives engaged in decision making are inherently involved in collective
leadership. Regarding the first issue, a collective is a sizable, higher-level entity, defined as a clustering of individuals larger
than a group, whose members are interdependent based on a set of shared expectations or a hierarchical structure (Dansereau
et al., 1984). Collectives can be composed of, for example, groups of groups, departments, functional areas, strategic business
units, organizations, alliances of organizations, and even industries (Yammarino & Dansereau, 2009; Yammarino et al., 2005).
Although collectives are often confused with groups, there are four main differences between them: size, expertise, level of interaction,
and the number of one-to-one connections, wherein the latter two aspects define the nature of interdependence, which is
the key difference between groups and collectives. The collective’s nature of interdependence is characterized by how its members
are organized within it and the type of relationship that exists between them. Within a collective, the relationship, or
interdependence, between members is weaker and members act more independently than they would in a group (or dyad).
Regarding the aforementioned second key distinction regarding collective decision making and leadership, Dansereau et al.
(1999) describe different ways in which levels of analysis can change over time. When a collective is formed, individuals come
together and move up a level to the whole collective level of analysis. This process is not just the accumulation of people until
the cluster is large enough to be considered a collective. Rather, the shift involves a change in the entity’s level of analysis wherein
those in the collective undergo an alteration in their frame of reference. Instead of seeing themselves as individuals with their
own wants, needs, and goals, they begin to see themselves as part of a collective and are concerned with accomplishing its
goals (Yammarino & Dansereau, 2009).
Although a collective can form without a leadership influence, in the majority of cases this referent shift is largely attributed to
the efforts of the leader(s) and his/her/their ability to bring together and unite individuals with a common purpose, task, or set of
expectations. Leadership likely helps establish the interdependence and connections among individuals who form the collective.
Moreover, it may be unlikely that such a collective would form entirely of its own accord without the influence of leadership,

which would bring individuals together and unite them under the goal of the collective (see Bass, 2008), often using the leader
skill of inspiration to do so. Leadership can manifest itself in different ways within a collective. It may be provided by a formal
leader, external to the collective and responsible for determining the task and the rules to perform it. In a leaderless collective,
members of the collective make decisions on their own. Similar to leaderless groups described by Bass (1949), a leaderless collective
will divide the necessary tasks among collective members, with each task being performed by a single member, multiple
members, or not addressed at all. By taking up these tasks, members of the collective provide structure where there initially is
none, which is a prominent leader behavior (Bass, 1954). As multiple leaders perform leadership tasks, traditional leadership
theory focused on a single leader and his/her small group of followers becomes insufficient for such increasing complexity.
Thus, theoretical advancements that allows for potential interactions among multiple leaders and larger entities of interest are
necessary to understand how leaderless collectives engaged in decision making are inherently involved in collective leadership.
Since scant prior research exists which attempts to understand collective decision making within the organizational behavior
and leadership field, there is limited theoretical framework from which to adopt constructs. As such, the constructs employed in
the current model come from related, more established literatures surrounding leadership and decision making, which may most
logically be connected to the notion of a collective and the process a collective uses in decision making. For example, these include
individual characteristics (i.e., intelligence and knowledge), collective characteristics (i.e., collective intelligence, participative leadership
type, and inspiration), work structure (i.e., collaboration method and mutual reliance), and task characteristics (i.e., task
complexity). Incorporating all of the above constructs, a proposed model of collective decision making for testing is summarized
in Fig. 1.
Overall, this research contributes to the leadership literature in three significant ways: (1) expanding and integrating research
in the fields of decision making, leadership, and collective intelligence, while extending the combined line of leadership and
decision making research to the collective level of analysis, and (2) introducing the organizational behavior and leadership field
to the construct of collective intelligence, a key aspect of collective decision making, and (3) providing a test of the proposed
collective decision making model through the use of multiple methods, i.e., agent-based simulations and a field study.
Conceptualization and hypotheses development
Collectivistic leadership
Collectivistic leadership looks at multiple individuals assuming leadership roles within dyads, groups, teams, and collectives
(Shuffler, Salas, Yammarino, Serban, & Shirreffs, 2012; Yammarino, Salas, Serban, Shirreffs, & Shuffler, 2012). Multiple theories
within this field offer perspectives on how multiple individuals can simultaneously share leadership: teams and multi-team
systems based leadership (e.g. Burke, DiazGranados, & Salas, 2011; Burke et al., 2006; Day, Gronn, & Salas, 2004; Kozlowski,
Gully, Salas, & Cannon-Bowers, 1996; Mathieu, Marks, & Zaccaro, 2002), network theory based leadership (e.g.Balkundi &
Harrison, 2006; Balkundi & Kilduff, 2005), shared leadership (e.g.Carson, Tesluk, & Marrone, 2007; Pearce & Conger, 2003;
Pearce, Manz, & Sims, 2008), complexity leadership (e.g.Marion & Uhl-Bien, 2001; Uhl-Bien & Marion, 2009; Uhl-Bien, Marion,
& McKelvey, 2007), and collective leadership (e.g.Friedrich, Vessey, Schuelke, Ruark, & Mumford, 2009; Yammarino et al.,
2010a, 2010b).
Collectivistic leadership is the result of several dynamic processes in which there is no singular path from which collectivistic
leadership will emerge. Therefore, there is no singular approach to collectivistic leadership that is appropriate. These theories can
be viewed as different processes through which leadership can be distributed among multiple individuals (Yammarino et al.,
2012), and a leaderless collective may use any of these different processes as they divide and perform the actions of a leader.
The leaderless collective prioritizes and/or determines what task(s) will be addressed, in part governed by the rules of the system
within which they are working and the norms developed by the collective. Thus, the collective will undertake task work using a
collective initiative to reach decisions. This prioritization and task development in consideration of various restrictions within the
system and collective itself resembles a leadership process at a lower level of analysis, where an individual leader may take a
similar approach to prioritizing and assigning task work to reach decisions. As such, leaderless collectives engaged in decision
making are inherently involved in a form of collective leadership distributed across the collective. A primary component of understanding
the distributed decision making process is collective intelligence.
Collective intelligence
Collective intelligence dates back to Aristotle, who first described the theory in Politics (Aristotle, 350 B.C.E). It has been
researched in many fields including psychology and sociology (Surowiecki, 2004), as well as computer science (Szuba, 2001),
and many different definitions have emerged for this concept (MIT Center for Collective Intelligence, 2010; Szuba, 2001;
Woolley, Chabris, Pentland, Hashmi, & Malone, 2010). In an organizational context, collective intelligence can be defined as the
phenomenon that occurs when a collective, acting as such, has greater intelligence than it would have had if its members were
acting as individuals or groups, where intelligence refers to an ability to solve problems.
Despite an existence of thousands of years, collective intelligence has never been formally studied in the field of organizational
behavior and leadership. However, many companies have discovered collective intelligence on their own and have already begun
to use it for categorizing, evaluating, and sharing knowledge, making predictions, and solving problems. Despite differences in the
objective for using collective intelligence, these methods make use of collective intelligence in a similar fashion. They consult

individuals, groups, or both individuals and groups within a collective directly or indirectly to accumulate and utilize the
collective’s knowledge. The accumulation of that collective’s knowledge depends on several key individual-level elements, including
individual intelligence and knowledge, which are discussed below.
Individual intelligence
Intelligence at the individual level, often referred to as general intelligence, or g, is defined as a “general mental capability that,
among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly
and learn from experience. It is not merely book learning…it reflects a broader and deeper capability for comprehending our
surroundings” (Arvey et al., 1994, p. A19). Intelligence has been a highly researched concept beginning with Spearman’s
(1904) work and has since been associated with many positive social advantages (e.g., economic self-sufficiency, employment,
educational achievement, affluence, lawful behavior, and legitimacy) (Herrnstein & Murray, 1994).
Perhaps its most important association is that with job performance. Campbell (1990) revealed general intelligence was a
significant determinant of performance for jobs that included information-processing tasks. Schmidt and Hunter (1998) later
corroborated these findings when they found it to be one of the best predictors of job performance, with the relationship even
stronger when the job was complex. In our model of collective decision making, the intelligence of each individual in the collective
contributes to the intelligence of the collective as a whole. This hypothesis may be somewhat definitional due to the nature of
collective intelligence, but it is necessary to assert to explain other relationships within the model.
Hypothesis 1. Individual intelligence will be positively related to collective intelligence.
Individual knowledge
Knowledge is the organized facts and principles within an individual’s mind pertaining to the characteristics of objects in a
given domain (Fleishman & Mumford, 1989). It should not be confused with the mere accumulation of facts (Chi, Glaser, &
Rees, 1982; Gettinger & White, 1979; Snow & Lohman, 1986) as it goes beyond accumulation to interpret and organize small
pieces of information into meaningful categories. These knowledge structures are formed through training, education, and experience
(Mumford & Connelly, 1991) and can include both codified and tacit knowledge. Codified (explicit) knowledge can be
explained in words or shown with drawings or writing. Tacit knowledge cannot easily be conveyed to another person and it
involves movement skills, physical experiences, senses, rules of thumb, and intuition (Nonaka & von Krogh, 2009). These knowledge
structures can also include information stored in both explicit and implicit memory, which include information conscientiously
and unconscientiously committed to memory, respectively (Roediger, 1990). Individuals with greater knowledge have
more extensive and diverse knowledge structures in which they organize information based on underlying principles and have
the ability to efficiently store this information. Greater knowledge is associated with greater problem solving capabilities
(Mumford & Connelly, 1991).
Within a collective the concept of intelligence refers to the collective’s ability to solve problems. At the individual level knowledge
is correlated with problem solving. This relationship is expected to hold at the collective level, so that the knowledge of each
individual within the collective contributes to the collective’s problem solving ability, or collective intelligence. This hypothesis
may also be somewhat definitional due to the nature of collective intelligence, but it is necessary to assert to explain other
relationships within the model.
Hypothesis 2. Individual knowledge will be positively related to collective intelligence.
Mutual reliance
Despite researchers often describing independence and interdependence as two completely separate concepts, they are actually
two ends of the same continuum (Stewart, 2006). Independence describes situations where individuals are not dependent on
each other and act as individuals (Bertucci, Johnson, Johnson, & Conte, 2011), while, at the other end of the spectrum, interdependence
describes situations where individuals depend on each other. In most cases, it is also the reason a collection of individuals is
formed (Campion, Medsker, & Higgs, 1993). The concepts of interdependence and independence have been compared to groupand
individual-level tasks, respectively, as interdependence results from cohesive behaviors and independence results from
fragmented behaviors (Cummings, 1978). We term this continuum mutual reliance, with high levels of reliance indicating interdependence
and low levels reflecting independence.
Collectives, as compared to groups, tend to have lower levels of mutual reliance (members are more independent). This difference
is a result of the large size of a collective and its communication structure, which may not allow or only allow limited
communication. Mutual reliance may also vary based on task. Some tasks inherently require individuals in a collective to work
together closely (e.g., writing a single, cohesive Wikipedia page), while others do not. Despite task-based variation in mutual
reliance, the collective will still likely maintain a level of mutual reliance lower than the typical group given their lower level
of interaction and the reduced number of one-to-one connections (Yammarino & Dansereau, 2009), which defines the nature
of interdependence, the key difference between groups and collectives. Independence is beneficial to a collective as members

use their own knowledge without being overly influenced by others. When members influence each other too much, mistakes can
become correlated and individual judgment errors may destroy the collective judgment (Surowiecki, 2004).
Hypothesis 3. Mutual reliance will moderate the relationship between individual intelligence and collective intelligence, such that the
relationship between them is stronger with lower levels of mutual reliance and weaker with higher levels of mutual reliance.
Hypothesis 4. Mutual reliance will moderate the relationship between individual knowledge and collective intelligence, such that the
relationship between them is stronger with lower levels of mutual reliance and weaker with higher levels of mutual reliance.
Collaboration method
Collaboration method (face-to-face vs. virtual) is a distinguishing feature of a collective. Due to a collective’s size, successful
face-to-face collaboration can be extremely difficult and time consuming, as it requires a predesigned method and considerable
manpower to implement. A physical space is required and physical collaboration materials must be purchased. Virtual methods
allow greater numbers of individuals to successfully collaborate within a collective from varied locations and on their own time
(e.g., asynchronously) with very little cost (Iandoli, Klein, & Zollo, 2013; Serban et al., 2015). Virtual groups and collectives do
not need to expend the time and effort to facilitate communicating with a large clustering of individuals face-to-face and can
therefore devote time and effort to collective decision making. This ease of collaboration will positively affect the level of
collective intelligence the collective is able to attain.
Virtual collectives also have the ability to collaborate both synchronously and asynchronously, during which different types of
participation occur. Synchronous collaboration fosters intense interaction, while asynchronous collaboration fosters more reflective
participation (Hrastinski, 2008). As face-to-face collaboration must be done synchronously, the resulting collaboration will
not have the benefit of reflective participation, which may offer virtual collaboration an additional advantage, contributing to a
higher level of collective intelligence.
Hypothesis 5. Collaboration method will moderate the relationship between individual intelligence and collective intelligence, such that
the relationship between them is stronger with virtual collaboration than with face-to-face collaboration.
Hypothesis 6. Collaboration method will moderate the relationship between individual knowledge and collective intelligence, such that
the relationship between them is stronger with virtual collaboration than with face-to-face collaboration.
Collective decision quality
For a collective tasked with making decisions, decision quality is the collective’s performance criterion or metric. Decision quality
is often described by the two dimensions of accuracy and speed (Gilliland & Landis, 1992; Howell & Kreidler, 1963). In the
current study we focus on accuracy, which is likely more critical than speed for collectives. Accuracy is defined as whether an
answer is correct or not. The types of decisions that need to be made by a collective often have no one correct answer. Therefore,
quality will be defined as the extent to which the decision made satisfies all of the necessary facets of the task. One of the principal
antecedents of decision quality is cognitive capabilities (Amason, 1996). It has been argued that a collection of individuals is
more effective at solving problems, particularly complex ones, when among them they possess a variety of knowledge (Bantel &
Jackson, 1989). Kraiger and Wenzel (1997) argue that it is not only the breadth of the knowledge possessed, but also the depth
that is important in decision quality. There is both great depth and breadth in a collective’s pool of knowledge to which each
member has contributed and a large amount of collective cognitive capabilities. The pool of knowledge which exists within a
collective and contributes to the collective’s cognitive capability, which we refer to as collective intelligence (as previously
defined), should enable the collective to make better quality decisions.
Hypothesis 7. Collective intelligence will be positively related to collective decision quality.
Participative leadership type
Leadership is a vast and diverse field; attempting to discuss each traditional leadership style (i.e., of a single leader) relative to
collective intelligence is nearly impossible (Bass, 2008; Yukl, 2009). Leadership here will be discussed in what appears to be the
most logical point in the model given the literature and the way collectives and collective intelligence appear to work in terms of
a continuum of participation in decision making. Participation ranges from autocratic or authoritative leadership (where the
decision is made solely by the leader), to consultation with subordinates (where subordinates can contribute their opinion), to
joint decision making with subordinates (where the leader allows the collective’s decision to be their own) (Durham, Knight, &
Locke, 1997; Vroom & Yetton, 1973).
There are three different ways in which a collective can come to a decision. As the perspective that leaderless collectives
engaged in decision making are inherently involved in collective leadership is taken here, these three decision making methods
are analogous with leadership styles using varying levels of participation. The first option is to have the individuals provide

information and change the deliverable until they reach consensus on a final version. This is the highest level of participation. The
second option is independent contribution in which each individual in the collective puts forth their own deliverable. This is
characterized by a lack of aggregation, as there is never a single deliverable to which all individuals in the collective have contributed;
the best deliverable is chosen from all deliverables by a leader figure. This is the lowest level of participation. The third
option is contribution aggregation, in which all deliverables can in some way be summarized to provide one final deliverable. Aggregation
is usually mathematically based and the deliverables provided by the individuals must be in some way numeric. This
exemplifies a medium level of participation.
Participative leadership is defined as joint decision making or shared influence in decision making by a leader and his/her
respective followers (Koopman & Wierdsma, 1998), with most research viewing participative leadership as generally homogeneous
within a workgroup (Somech, 2003). Cognitive models of participative leadership suggest participation in decision making
by followers enhances the flow of knowledge and encourages the use of important information in organizations (Miller & Monge,
1986). It is proposed that decisions will be made with better pools of knowledge using participative leadership, as workers have
greater knowledge of their work than do managers (Anthony, 1978; Frost, Wakely, & Ruh, 1974). Depending on the degree to
which subordinates already possess or can acquire relevant information to the task and communicate it with other members of
the collective, benefits of participative decision making may lie in the cognitive realm Scully, Kirkpatrick, and Locke (1995)).
Participative decision making may be the needed link in enhancing information exchange with a collective (Durham et al.,
1997) which would ultimately enhance decision quality by allowing the decision to be made with the maximum amount of available
knowledge.
Hypothesis 8. Leadership style will moderate the relationship between collective intelligence and collective decision quality, such that
the relationship between them is stronger with more participative style of leadership and weaker with less participative styles of
leadership.
Inspiration
Inspiration is the suggestion, arousal, or creation of some feeling or impulse within a person (Thrash & Elliot, 2003) and is
composed of three core characteristics: (a) transcendence – the orientation of an individual toward something more important
than their usual concerns, (b) evocation – when the individual does not feel directly responsible for becoming inspired, and
(c) motivation – when the individual expresses what he/she was inspired to do (Thrash & Elliot, 2003). Inspiration, as a leader
skill, is relevant to collective decision making (see Bass, 2008) not only in the formation of a collective, but also in the relationship
between collective intelligence and decision quality. It can be used as a motivational technique where powerful emotional
responses are evoked from followers. These emotional responses energize them to exert extra effort in completing the task at
hand (Sosik & Dinger, 2007). With an increased level of determination and effort, the relationship between collective intelligence
and decision quality can be stronger.
Hypothesis 9. Inspiration will moderate the relationship between collective intelligence and collective decision quality, such that the
relationship between them is stronger when the leader is inspiring and unaffected when the leader is not inspiring.
Task complexity
Research on tasks is often concerned with task type, and complexity is often used as its distinguishing feature. Many lines of
research have studied task complexity: information-processing and decision-making (MacCrimmon, 1976; Schroder & Suedfeld,
1971), task and job design (Hackman, 1969a, 1969b), and goal-setting (Campbell & Gringrich, 1986; Frost & Mahoney, 1976).
Task complexity has been examined from three perspectives, as a primarily psychological experience, an interaction between
person and task characteristics, and a function of objective task characteristics (Campbell, 1988). Our research adopts the latter
perspective, which seems to be the most straightforward approach relevant to collective decision making.
Task complexity has been described in many ways, one of the most simple being a task placing a high cognitive demand on
the individual trying to perform it (Campbell & Gringrich, 1986). Campbell (1988) presents a framework for categorizing task
complexity types based on Schroder, Driver, and Streufert’s (1967) properties of a complex task. He presents four basic task
characteristics that increase the complexity of a task: the presence of multiple potential paths to arrive at a desired outcome,
multiple desired outcomes to be attained, conflicting interdependence among paths to multiple outcomes, and uncertain or probabilistic
links among paths and outcomes (Campbell, 1988). Based on different combinations of these four task characteristics, five
distinct types of tasks are outlined: simple, decision, judgment, problem, and fuzzy.
Campbell (1988) describes decision and fuzzy tasks as complex tasks that tend to be particularly difficult to solve because of
the multiple desired end-states that they entail. The decision maker is trying to optimize a decision between different wants or
needs and no clear answer is able to be discerned as the correct answer. The decision maker must use whatever information is
available to subjectively decide on the best option (Campbell, 1988). Due to the possibly biased nature of decisions with multiple
desired end-states, they are susceptible to factors (e.g., emotions) other than clear facts, which can negatively affect decision
quality.

Complex tasks are inherently more difficult to solve than simple ones, wrong choices can be made and thus decision quality
can be affected. In some cases there will be examples of collective stupidity rather than collective intelligence. Collectives are no
more immune to this concept than are teams or individuals. They are, however, more insulated from it. Researchers suggest that
having a greater number of people involved with decision making is appropriate with complex tasks (Campbell & Gringrich, 1986;
Vroom & Deci, 1960), particularly when utilizing a greater number of individuals allows decision makers to accumulate more
knowledge than they would have working on their own (Scully et al., 1995). The greater pool of knowledge that a collective
contains relative to a group, insulates the decision quality from complex tasks to a small degree in that decisions are made
with the decision maker(s) having more complete knowledge, but ultimately decision quality will be negatively affected by
task complexity.
Hypothesis 10. Task complexity will moderate the relationship between collective intelligence and collective decision quality, such that
the relationship between them is weaker with more complex tasks and unaffected with less complex tasks.
Method
Multiple methods
Collective decision making and leadership is not a research area particularly conducive to study. The sheer size of a collective
and the necessity to have multiple collectives to perform statistical analyses makes it extremely difficult to study collectives in a
traditional experimental setting or within a field study. One possible solution to this dilemma is to use agent-based modeling to
simulate collective decision making. Agent-based modeling is a simulation technique which enables modeling a system composed
of autonomous agents that interact with one another and their environment for a specified number of iterations (Macal & North,
2010). The system dynamics in these simulations are guided by mathematical equations or if-then statements, which stipulate the
rules for how agents within the system will behave (Vancouver, Tamanini & Yoder, 2010; Vancouver, Weinhardt & Schmidt,
2010). Examples of agent-based modeling can be found in various disciplines including economics, epidemiology, warfare
(Macal & North, 2010) and organizational behavior and leadership (Dionne & Dionne, 2008; Dionne, Sayama, Hao, & Bush,
2010; Serban et al., 2015).
Agent-based modeling has three main elements called agents, agent relationships, and the environment (Macal & North,
2010). These components are generated by the simulation using practical recommendations and existing theory, such as individual
intelligence parameters, based on results from the Wonderlic Personnel Test for an average population (see below). Using the
simulations capability of approximating individuals who could exist in the population in question, their interactions, and the
environment in which they exist, researchers are able to examine the phenomena outlined in the simulation without spending
the time and money necessary to collect data from thousands of individuals and hundreds of collectives. These unique characteristics
make agent-based modeling well suited to study areas difficult to access in the real world (e.g., a collective engaging in leadership
and making a decision).
However, agent-based modeling on its own is not a substitute for traditional methods. Supporting results from a simulation
provide necessary but not sufficient evidence to corroborate a theoretical model (Kozlowski, Chao, Grand, Braun, & Kuljanin,
2013). They should be viewed as complements to traditional methods, as they allow researchers to develop guiding principles
for necessary subsequent empirical tests (in lab or field studies) (Epstein, 1999; Goldspink, 2002). However, this brings the argument
full circle, as we are once again confronted with the obstacle of collectives being difficult to study within a traditional
experimental setting or a field study due to their size. Thus, to best test the proposed conceptual model, while also balancing
the advantages and disadvantages of the methods discussed, both agent-based simulations and a modestly sized field study are
employed.

A field study on collectives of the size presented here (described below) is simply not large enough to permit traditional multilevel
analysis which would allow for a direct comparison between collectives. Instead, this study summarizes the field data
obtained and presents it in as many ways as possible given the size and scope, while focusing primarily on within-collective
processes, issues, and outcomes. Given that collective decision making is a new and primarily theoretic area of study in organizational
behavior and leadership, even a small field data set on collectives, beyond agent-based simulations, is valuable and can offer
a first empirical glimpse into real world collectives engaged in leadership and making decisions.
Method 1: Agent-based modeling simulation
This study includes a simulation of the full conceptual model of collective decision making, where a collective will be formed
and tasked with making a decision. The simulation is developed using the Python programming language (http://www.python.
org/) using parameter values and conceptualizations based on practical recommendations and existing theory. Consistent with
initial explorations of collective decision making, parameters were established for high and low levels of each of the six simulation
variables. Table 1 includes the means and ranges for their values.
Task characteristics
Task complexity
Task complexity is defined as a randomly generated number between 20 and 100, with 60 representing the cutoff between
low and high task complexity. This range was arbitrarily chosen in order to make the numerical values more interpretable.
Task complexity serves as an input when the true problem function, or correct answer, is generated to determine the length of
the problem domain. Each point in the problem domain represents a decision that the collective will have to make. Lower levels
of task complexity, which will create tasks requiring fewer decisions, are associated with easier tasks, while higher levels of task
complexity, which create tasks requiring more decisions, are associated with more difficult tasks.
Collective characteristics
Inspiration
Inspiration is defined as a randomly generated number between 0 and 1, where 0.5 represents the cutoff between low and
high inspiration. Inspiration is a single value assigned to the entire collective, which represents how inspiring the collective’s leader
is.
Collaboration method
Collaboration method (among collective members) is defined as a randomly generated number between 0 and 1, where 0.5
represents the cutoff between collaboration based primarily on virtual methods (values below 0.5) and collaboration based
primarily on face-to-face methods (values above 0.5).
Mutual reliance
Mutual reliance (among collective members) is defined as a randomly generated number between 0 and 1, where 0.5
represents the cutoff between independence (values below 0.5) and interdependence (values above 0.5).
Collective size
Collective size is defined as 50 members. This number was chosen as a starting size because it is significantly larger than the
typical group or team, which maximizes around 10 members (Likert, 1977), but it is also small enough to ensure that run time for
the simulation would not be exorbitant. It also approximates the size of the actual collectives examined in the field study (see
below) allowing for more direct parallels to be drawn between the simulation and field study results.
Individual characteristics
Intelligence
Individual intelligence is defined as a number between 0 and 50, randomly generated from a normal distribution with a mean
of 16.75 for individuals in collectives with low levels of intelligence and a mean of 26.75 for individuals in collectives with a high
level of intelligence. Both low and high intelligence collectives have a standard deviation of 7.6. These values are based on the
range and distribution in an average population as described by reports from the Wonderlic Personnel Test where scores are
between 0 and 50 with a mean of 21.75 and a standard deviation of 7.6 (Wonderlic Personnel Test, Inc., 1992). Taking into
account the necessity to distinguish between high and low intelligence collectives, the means in each have been shifted up or
down appropriately.
Knowledge
Knowledge of an individual is defined as a randomly generated number between 0 and −0.5, where −.25 represents the cutoff
between low and high knowledge. It represents the negated absolute value of the difference between the individual’s problem

function and the true problem function at each point in the problem domain, such that a higher value of the variable means a
higher amount of knowledge. The highest level of knowledge would be when the individual’s problem function and the true problem
function are identical, which would be represented with a value of 0. Values above −0.25 are representative of individuals
who have high knowledge, while values below −0.25 are representative of individuals who have low knowledge. Each individual
will have a knowledge value for each point in the problem domain, which is averaged to determine the individual’s overall
knowledge.
Simulation algorithm
In the first stage of the simulation, the task, collective, and individuals are simulated. Task complexity is assigned by randomly
choosing a value from the set distribution already defined. This value is then used to generate the true problem function
(i.e., correct answer) such that the length of the problem domain is equal to task complexity. Random numbers between 0 and
1 are then assigned to several sample points in the problem domain and the remaining values are generated by interpolating
between those points. Since each point within the problem domain represents a choice and a value to be optimized, for more
complex tasks there will be a greater number of decisions to be made. Fig. 2 depicts an example of a true problem function.
Next, the collective characteristics of inspiration, mutual reliance, and collaboration method are assigned by randomly choosing
a value from each variable’s set distribution and 50 individuals are created. Connections between individuals are generated based
on the values assigned to inspiration and mutual reliance. The higher the level of inspiration, the more the members of the
collective will attempt to form connections with one another. The higher the level of mutual reliance, the more likely the members
of the collective will form a connection if the opportunity presents itself. Each individual in the collective is then assigned the

characteristics of individual intelligence and knowledge by randomly choosing a value from each variable’s set distribution. An
individual problem function is then generated by adding or subtracting the individual’s knowledge to the value at each point
on the true problem function.
In the second stage of the simulation, individuals communicate to discuss their answers and come to a consensus on the problem
function. First, a speaker is chosen at random from among the individuals in the collective. Then a topic for the speaker to
discuss (i.e., a position in the problem domain) is chosen randomly based on the speaker’s intelligence, where the more intelligent
he/she is, the more likely he/she is to choose a topic in which he/she has higher knowledge. The speaker then shares his/her opinion
on the selected topic (i.e., the value he/she has for the problem domain position selected). Each of the speaker’s neighbors
evaluate the shared opinion and update their own to move slightly towards that of the speaker with a small perturbation in
that value based on noise. Noise is generated based on the collaboration method of the collective such that face-to-face collectives
experience a higher level of noise than those using predominantly virtual communication, which reflects the face-to-face
collective’s ability to only collaborate synchronously and the resulting participation limitation.
After the collective reaches consensus the results of the simulation are evaluated. The first value determined is collective
intelligence. At current, there is no accepted measure of collective intelligence, but there are many different ideas for how to measure
it that may capture the essence of the construct (e.g. MIT Center for Collective Intelligence, 2010; Szuba, 2001). The following
are three different possible methods for quantifying collective intelligence. In the first method, the intelligence and knowledge of
each member is averaged and a collective intelligence measure is generated by the following formula:
ðAverage Intelligence−50Þ  Average Knowledge ð1Þ
Using this formula, the highest amount of collective intelligence would be represented as 0. The second and third methods
both also make use of the same formula, but with smaller samples. The second method examines the average of the top five
intelligence and knowledge scores for the collective, while the third method examines the average of five members selected at
random.
Next, problem functions are calculated for the three different decision making types of a collective, where each is representative
of a different level of participative leadership. Using simulation each collective can be examined as if all three decision making
types had been used simultaneously to make the same decision. Consensus decision making represents the highest level of participation
by the members of the collective in decision making. The consensus problem function is determined by the problem
function on which all of the individuals in the collective decided when sharing information and coming to consensus. Contribution
aggregation decision making represents a medium level of participation by the members of the collective in decision making. The
contribution aggregation problem function is determined by averaging the values from each individual’s original problem function
at each point in the problem domain. Independent contribution decision making represents the lowest level of individual participation.
The independent contribution problem function is determined by examining the original individual problem function of
each member of the collective and calculating its quality.
Quality is measured by calculating the sum of the square distance between the proposed problem function and the true problem
function over the entire problem domain, which is then multiplied by −1. Using this method, the distance between the
actual answer and the proposed answer is examined, ignoring whether the error is above or below the correct answer. The maximum
value of collective decision quality is 0, which would occur when the proposed problem function is identical to the true
problem function. The larger the distance between the two values (i.e. the more incorrect the proposed answer is), the more

negative the value of decision quality. The function with the highest quality is selected as the independent contribution problem
function. Examples of the selected problem function in comparison to the true problem function for each of the decision making
types can be seen in Fig. 2.
Collective decision quality is then calculated for problem functions chosen by contribution aggregation and consensus using
the same measure of quality as described above. There is no need to calculate the collective decision quality for the independent
contribution problem function, as it was selected based on its quality and the appropriate value was recorded at that time. A
summary of the simulation algorithm is shown in Table 2.
Using the algorithm as described above, a Monte Carlo simulation was conducted. This technique makes use of random
sampling distributions by repeating the sampling process many times, allowing for system properties to be averaged over the different
generated states (Dionne & Dionne, 2008; Dionne et al., 2010; Metropolis, Rosenbluth, Rosnebluth, Teller, & Teller, 1953).
This established technique is widely used in physical sciences and mathematics (Binder & Heerman, 2002; Caflisch, 1998). To test
the 10 hypotheses, the simulation was conducted using parameter settings of low and high for individual intelligence, individual
knowledge, task complexity, collaboration method, mutual reliance, and inspiration. This produced a Monte Carlo with
2x2x2x2x2x2x20=1,280 total simulation runs, such that 20 representative collectives were generated for each possible combination
of parameter settings. The number of 20 representative collectives was chosen based on preliminary analysis which showed
that no additional information would be provided by adding any additional collectives.
Method 2: Field study
The data presented in this study were obtained from a public research university in the Northeast United States as part of a
strategic planning effort which involved nine collectives making decisions over a three month period. During that time, the
collectives met both face-to-face and virtually, using an online project management website. The goal was to develop a strategic
plan for the university for the next twenty years focusing on nine strategic areas. Nine collectives were formed and each given a
strategic area focus and charge (i.e., assignment) that included Student Success, Rankings and Reputation, Philanthropy, Infrastructure,
Global Engagement, Diversity and Inclusiveness, Creative Activities and Research, Community Engagement, and Advanced
Learning. The collectives were comprised of faculty, staff, community members, students (undergraduate and graduate), and
alumni who were either nominated or volunteered to help with this project. Each collective was comprised of approximately
50 members for a total of 455 individuals. Within each collective, two to three members were appointed by the university
president as co-chairs of the collective.
The collectives met weekly to develop a vision statement for the university within their assigned strategic priority for five and
20 years into the future. These collectives were tasked with developing projects that would help the university achieve the goals
they had set forth. In total 176 different proposals were submitted by the nine collectives and evaluated by the steering committee,
comprised of university leaders and the co-chairs of each of the collectives. Each member of the steering committee evaluated
each proposed project in the areas of impact and priority on a scale of one to five. The proposals were then ranked according to
the sum of their average impact and priority scores, discussed by the committee, and 46 proposals were selected for implementation.
For each of the selected proposals, an implementation plan was developed which included major milestones and a budget
was created for completing the project.
To study the collective decision making process of the strategic planning effort collectives, qualitative content analysis was
employed using a directed approach, in which coding schemas were developed from theory and applied to the data (Parry,
Mumford, Bower, & Watts, 2014). Two experienced and trained coders working independently to code the source materials
had 96% initial agreement and 100% agreement after discussion. Source materials which were content coded came from the
project website, a project management website used by members (called Basecamp), and documentation provided by the administrators
of the program.
The project website included detailed information about each member of each collective including their current position(s),
their office(s), department(s), or organization(s), their affiliations with the university, and their major(s) (where applicable for
students and alumni). It also provided more general information on the project detailing its implementation and which projects
were eventually selected for the university to pursue. From the project administrators, scoring of the proposals was obtained, and
broken down by impact and priority, as evaluated by each member of the steering committee. The project management website

that the collectives used allowed the members of each collective to collaborate virtually in addition to the weekly meeting where
collectives collaborated face-to-face, while keeping a record of each discussion, as well as a record of any documents the members
shared with each other. Table 3 includes a summary of each coded field study variable.
Variables
Individual intelligence
Given that the project was completed prior to this study and there was no access to the participants, individual intelligence
could not be measured directly with standard survey measures such as the Wonderlic Personnel Test. Instead, a surrogate measure
for intelligence was used, based on the affiliation of the individual to the university and obtained from the project website.
Affiliations were broken into three categories. The first contained those collective members listed as faculty. These individuals
would need a high level of intelligence to complete their doctorate degrees and thus were assigned the highest intelligence
value of 1.0.
The second category contained students, who were assigned the lowest intelligence value of 0.5, given that less intelligence is
necessary to obtain an undergraduate or master’s degree, than a doctorate degree. The third category contained alumni, community
partners, industry partners, and staff members. Individuals in this category were assigned a medium intelligence value of
0.75, as most of them had an undergraduate and/or master’s degree as well as additional life experience given their time in
the work-force, which merited them more intelligence than students but less than the faculty. When an individual had more
than one affiliation (i.e., they had both attended the university as a student and now work there as a staff member), they
were assigned the highest value possible given their multiple affiliations. The average of all individual intelligences of the members
of the collective were assigned to the collective.
Individual knowledge
To capture the individual knowledge of each member in the collective, the number of departments, majors, offices, and schools
with which the individual was affiliated through their work or their schooling was used as a measure of the breadth of their
knowledge, obtained from the project website. Given the information obtained on each individual, there was no way to measure
the depth of their knowledge directly. The range of their knowledge was thus approximated by counting the number of areas they
had worked or learned. Each individual had between one and five areas and was assigned an individual knowledge value accordingly.
An average of individual knowledge for each individual in the collective was assigned to the collective.
Collaboration method
To measure virtual collaboration, the number of discussion posts were counted for each non-leader in each collective, based on
the recorded discussions on Basecamp. Each individual had between zero and 68 discussion posts. An average of these discussion
counts for each individual in the collective was assigned to the collective as the degree of virtual collaboration.
Mutual reliance
To measure mutual reliance, the number of authors of the proposals that the collective submitted were examined, based on
proposal documents on Basecamp. When mutual reliance is low, the collective works mostly independently and there would
be a lower number of authors on each proposal. When mutual reliance is high, the collective works more interdependently
and there would be a higher number of authors on each proposal. Mutual reliance for the collective was recorded as the average
number of authors listed on the collective’s proposals. Proposals which did not list authors were not included in the measure.
Collective intelligence
The measure of collective intelligence was obtained by averaging the sum of the impact and priority scores given to all of a
collective’s proposals by the steering committee. By using a measure that captures the impact and priority of the collective’s submitted
proposals, the ability of the collective to solve the task at hand is captured, which is at its core collective intelligence.
Task complexity
To measure task complexity the collective’s charge and vision statement were reviewed, both of which were posted documents
on Basecamp. The charge outlined the strategic area assigned to them and what their task would be for the length of
the project. The vision statement was a collectively generated document outlining the goals for the university within their respective
strategic initiative. The coding scheme for task complexity applied to these documents was based on Campbell’s (1988) work
on the four aspects of task complexity described above. The tasks were scored such that highly complex tasks received a 1.0, tasks
of medium complexity received a 0.5, and simpler tasks received a 0.0.
Participative leadership style
There are three different decision making types of a collective, including independent contribution, contribution aggregation,
and consensus. Given the tasks the collectives in the strategic planning effort project had to complete, it was not possible for
them to make such decisions using contribution aggregation, as their proposals and solutions could not be summarized mathematically.
However, the collectives did have the option of deciding by consensus or by independent contribution. To determine
how the collective made decisions, leader language in discussion board posts was analyzed. Collectives making decisions by

independent contribution would have language that detailed individual assignments and emphasized individual contributions to
decision making. Collectives deciding by consensus would have language that detailed the collective assignment and emphasized
the collective contribution towards completing the assignment. Unfortunately for research purposes, every collective in the sample
used consensus as a method of making decisions. Due to lack of variability, the measure for participative leadership style could
not provide any insight to collective decision making and was not used in any analyses.
Inspiration
The number of discussion posts (recorded on Basecamp) for each co-leader in each of the collectives was used as a surrogate
measure for inspiration, since it was impossible to access participants and measure it directly. The number of co-leader discussion
posts served as an adequate surrogate, as it reflected the co-leader’s level of involvement in the process and the amount of effort
put forth into leading the collective. Each co-leader had between one and 44 discussion posts. An average of these discussion
counts for each of the two to three co-leaders in the collective was assigned to the collective as their degree of inspiration.
Collective decision quality
The measure for collective decision quality is the percentage of proposals submitted that were selected by the steering
committee for implementation. Collectives submitted between nine and 57 proposals each for a total of 176 proposals. From
these, a total of 46 were chosen for implementation, with between one and 11 of those proposals coming from each collective.
Percentage of selected proposals ranged between 10% and 44%.

Simulation results
Results of the tests of the hypotheses are shown in the figures. In all figures which feature results from the simulation, collective
intelligence is represented as an average of the three measurement methods of collective intelligence. They were represented
in this way, as there is no evidence at this time supporting one measurement method over the others and preliminary analysis
indicated a high level of agreement between the different measurement methods.
Within the simulation results, support was found for Hypothesis 1 and 2 (as shown in Figs. 3 and 4). Individual intelligence
and knowledge of each member of the collective was considered as a whole (pictured in blue) and separately (pictured in
green). The relationship between individual and collective intelligence was positive and had a line of best fit of y=0.0072x-
6.933 (r2=0.017). The relationship between individual knowledge and collective intelligence was positive with a line of best
fit of y=23.646x-0.159 (r2=0.749).
Support was not found for Hypotheses 3-6 which discussed moderation of these prior relationships by mutual reliance and
collaboration method (as shown in Figs. 5-8). Individual intelligence and individual knowledge of the collective are considered
as a whole. Here and in subsequent figures which have a moderating variable broken into high and low levels, simulation results
represent all 1280 collectives generated via the Monte Carlo technique split by level of the moderating variable such that 640
collectives have a low level (depicted in green), and 640 collectives have a high level (depicted in blue). Hypothesis 7 was supported
(as shown in Fig. 9), as collective intelligence and collective decision quality had a positive relationship with a line of best

fit of y=0.302x+0.376 (r2=0.528). In Figs. 9, 11, and 12, collective decision quality is represented as an average of the three
different measurements of quality generated for each collective. They were represented in this way so that generalizations
could be made across different decision making styles, where appropriate.
However, support was not found for Hypothesis 8-10 (as shown in Figs. 10-12) which discussed moderation of the collective
intelligence and collective decision quality relationship by participative leadership style, inspiration, and task complexity. In Fig.
10, all 1280 collectives generated by the Monte Carlo were included, but the collective decision quality values are split by participative
leadership type. The level of participative leadership indicates how the collective made their decision, with low levels of
participative leadership deciding by independent contribution (pictured in green), medium levels by contribution aggregation
(pictured in blue), and high levels by consensus (pictured in red).
Field study results
Within the results obtained from the field study, Hypothesis 1 and 2 (as shown in Figs. 3 and 4) examine individual intelligence
and knowledge of each member of the collective considered as a whole (pictured in blue). Support was found for
Hypothesis 1 with a positive relationship between individual and collective intelligence with a line of best fit of y=
0.115x+6.070. Support was not found for Hypothesis 2, which discussed the relationship between individual knowledge and
collective intelligence.
Support was not found for Hypothesis 3-6 which discussed moderation of the prior relationships by mutual reliance and
collaboration method (as shown in Figs. 5-8). Within these figures, individual intelligence and individual knowledge of the

collective are considered as a whole. Here and in subsequent figures which make use of a moderating variable broken into high
and low levels, the results from the field study represent all nine collectives from the strategic planning effort project, split by
level of the moderating variable, such that collectives with a level below the grand mean of the variable represent a low
level of the variable (depicted in green), and the collectives with a level which exceeds the grand mean represent a high
level of the variable (depicted in blue). While plots in Figs. 7 and 8 may seem counterintuitive for the field study results,
the measure for collaboration was based on virtual discussion and thus needed to be reverse-scaled. Hypothesis 7 was supported
(as shown in Fig. 9) with the relationship between collective intelligence and collective decision quality having a
positive relationship with a line of best fit of y=0.171x -0.745. Hypothesis 8 could not be examined in the field study results
due to a lack of variability.
Hypothesis 9 looked at the relationship between collective intelligence and collective decision quality, but included the moderator
inspiration (as shown in Fig. 11). The field study results describe this relationship with lines of best fit of y=0.135x-0.539
and y=0.252x-1.250, for low and high inspiration, respectively. The relationship between collective intelligence and collective
decision quality is nearly twice as strong for high as it is for low inspiration, so we can conclude that there is a difference in
the relationship between collective intelligence and collective decision quality when examining collectives solving problems
with varying levels of inspiration. For high inspiration the relationship is stronger, which aligns with the hypothesis. However,
for low inspiration the strength of the relationship actually decreases slightly when it was posited that it would be unaffected.
Therefore, Hypothesis 9 was partially supported by the field study results.
Hypothesis 10 examined the relationship between collective intelligence and collective decision quality, with task complexity
as moderator (as shown in Fig. 12). All collectives in the strategic planning effort project have been included in Fig. 12, but are

split by the level of task complexity. The field study results describe this relationship with lines of best fit of y=0.397x-2.034, y=
0.063x-0.096, and y=-0.139x+1.287 for low (depicted in green), medium (depicted in blue), and high (depicted in red) task
complexity, respectively. The differences between these lines of best fit are fairly large, so we can conclude that there is a difference
in the relationship between collective intelligence and collective decision quality when examining collectives solving problems
with varying levels of task complexity.
Hypothesis 10 suggests the relationship between collective intelligence and collective decision quality would be weaker with
more complex tasks and unaffected with less complex tasks. In examining just low and medium levels of task complexity,
Hypothesis 10 and the data are in agreement that the relationship between collective intelligence and collective decision quality
will be stronger under higher levels of inspiration, which in this case would be medium inspiration, than under lower levels of
inspiration which agree with the hypothesis. However, they are in disagreement in that the relationship with lower levels of
inspiration decreases in the results when Hypothesis 10 asserts it should be unaffected. The data also differs from the hypothesis
in that when task complexity is high the relationship becomes negative. Therefore, Hypothesis 10 was partially supported by the
strategic planning effort results.
Results across methods
Comparing the results from the simulation and the field study, both lend support to Hypothesis 1 and 7, neither set of results
lent support to Hypotheses 3–6 or 8, and only one of the methods was able to lend support to Hypothesis 2, 9, and 10. All the
simulation and field study results for the hypotheses are summarized in Table 4. That only three of 10 hypotheses, rather than
all the hypotheses, were supported by the simulation results, lends credibility to the “realism” of the simulation in relation to
field studies where such outcomes are typical.

Discussion
Key findings and implications
Given that collective decision making is a new area of study in organizational behavior and leadership, and predominantly theoretical,
this empirical research serves as a first glimpse into a previously unexplored area. A major contribution of this work is its
investigation of the relationships between individual and collective intelligence and collective intelligence and collective decision
quality, which were hypothesized as positive in Hypothesis 1 and 7 and were supported by the results from the agent-based simulation
and the field study. These relationships form the major backbone of the conceptual model asserted above, in which collectives
are engaged in leadership and decision making, and support for them should be seen as preliminary confirmation that
collective decision making is an area of research worth pursuing further.
The relationship between individual knowledge and collective intelligence, presented in Hypothesis 2, was supported only by
the simulations. This may be due to the surrogate method used to measure knowledge in the field study not being adequate to
capture the variable fully. As the field study was conducted after the collectives had completed their decision making task, it was
impossible to interact with the collective members to measure individual knowledge directly. Future research should explore
other individual intelligence measurement options.
Hypothesis 9 and 10 asserted the relationship between collective intelligence and collective decision making would be
moderated by inspiration and task complexity, respectively. These hypotheses were partially supported by the field study results.
The lack of support for Hypothesis 9 and 10 in the simulation may result from the implementation of inspiration and task
complexity. Inspiration was conceptualized as influencing the amount of effort members of the collective put into solving the
problem and into establishing connections between collective members. Inspiration could be interpreted differently however

(e.g., influencing the amount of effort put forth into discussion when the collective is coming to consensus). Task complexity was
conceptualized as the number of decisions that a collective would need to make to complete their task, where more complex
tasks required more decisions to be made. This measure operated under the assumption that all decisions were equally difficult
and did not explore the possibility that decisions could vary in terms of difficulty. Future research should explore these variations.
Hypothesis 3 and 5 looked at the relationship between individual and collective intelligence, moderated by mutual reliance
and collaboration method, respectively. Hypothesis 4 and 6 looked at the relationship between individual knowledge and collective
intelligence moderated by mutual reliance and collaboration method, respectively. These hypotheses were not supported by
the simulation or the field study results. Within the simulation, these results may be due to the way mutual reliance and collaboration
method were implemented. Mutual reliance is conceptualized such that the higher the level of mutual reliance, the more
likely the members of the collective will form a connection if the opportunity presents itself. An alternative conceptualization
could be that the connections between members of the collective are fixed and mutual reliance affects how often those connections
are utilized in making a decision. Collaboration method is conceptualized such that predominantly face-to-face collectives
experience a higher level of noise than predominantly virtual ones. An alternative conceptualization could be that collaboration
method affects the number of individuals involved in a single discussion, such that individuals using virtual methods can share
their opinion simultaneously with a greater number of people than they could face-to-face.
Within the field study, these results may be due to the way mutual reliance and collaboration method were measured. Mutual
reliance was measured by the number of authors on the proposals the collective submitted. Collaboration method was measured
using the number of discussion posts for each non-leader in each of the collectives, based on the recorded discussions on
Basecamp. These surrogate measures may not have captured the variable fully. Due to the field study being conducted after

the collectives had completed their decision making task, it was impossible to interact directly with the collective members to
measure these variables more directly. Future research should explore other measurement options.
Hypothesis 8 looked at the relationship between collective intelligence and collective decision making moderated by participative
leadership type. Due to a lack of variability in the participative leadership type variable in the field study, this hypothesis
could only be explored via simulation and no support was found. The explanation may reside in the way in which decision making
methods were operationalized. The best answer selected for independent contribution assumes that the choice is based on
perfect information and rational choice, which is not always the case in the real world. Additionally, with contribution aggregation

it was assumed that individuals were able to share their knowledge without communication error, which again is not always the
case. Future research should explore these variations.
Although time was not of interest within this work, it may be worth noting that the high quality of decisions achieved by consensus
came at the price of time. Consensus takes much longer than other methods, with individuals having to share their ideas
and convince others to change their mind. Simulation results indicate that contribution aggregation and consensus performed
substantially better than independent contribution in yielding high quality decisions. However, consensus outperformed contribution
by a narrow margin. Thus, when making time sensitive decisions, leaders may want to choose contribution aggregation when
individual decisions can be mathematically combined or averaged. Otherwise, the leader may opt to allow the discussion to only
last a set amount of time and then force the collective to make a decision based on their current opinion utilizing a technique
such as a majority rule vote in an attempt to harness the consensus decision making method’s quality but also to keep to a
time restriction.
Limitations and future research
As is the case with any research, there are limitations associated with this work that need to be acknowledged. First, the
models used are simplified in comparison to the actual process of decision making a collective undergoes and there may be
other factors involved in the process not accounted for in this model (e.g., another type of leadership such as ideological leadership,
which may have its own unique influence on the relationship between collective intelligence and decision quality). Given
this simplification, future research should incorporate additional constructs that may be relevant in the collective decision making
process.
Future research should also explore how variations in the size of the collective would impact the amount of individual intelligence
and knowledge infused into the collective, as well as examine non-linear relationships, which may reveal interesting
results. Moreover, as previously mentioned, there is no accepted way to measure collective intelligence currently. The three measures
used in this simulation and the surrogate measure used in the field study, are only a starting point. Future research should
explore better measures to capture collective intelligence.
Another limitation of this research, involves assumptions made in the development of the simulation and the parameters used.
The values employed were based on practical recommendations and existing theory, but should be viewed as merely a starting
point for further investigations. Simulation generated values are created based on patterns previously observed by the given
unit of analysis, but they are not directly associated with specific empirical evidence. Had the simulation been written with
different assumptions or conceptualizations of the parameters, it would have resulted in different findings and outcomes. Therefore,
interpretation of these findings is limited to the conditions represented within the simulation model. Future research should
explore differential starting points and their impact on the outcomes and results.
Limitations unique to the field study include the sample being comprised of only nine collectives. A field data set of this size
offers a first empirical glimpse into collective decision making, but is not large enough to permit traditional multi-level analysis
which would allow for direct comparison between collectives. An increase in sample size would allow for comprehensive
multi-level statistical testing. Alternatively, future research may consider building upon the current data set by collecting additional
information on new collectives completing a different task or determining a method to divide the current collectives into
multiple collectives (e.g., based on sub-missions or sub-goals).
Moreover, research should also attempt to collect data which includes, for all variables in this model, sufficient variability
to examine each variable critically. This was impossible here due to the lack of variability in participative leadership style.
Since the field study data was collected in a university setting, there was also low variability within individual intelligence
and knowledge compared to the variability within other work organizations or the human population in general. Researchers
should thus investigate samples representative of a broader range of individual intelligence and knowledge.
Last, future work should seek to collect data from collectives actively in the decision making process. With direct access
to both individuals in the collectives and their decision making, better measurement techniques (rather than the surrogates
used here) may be employed.
Conclusion
Overall, this research introduces the organizational behavior and leadership field to collective intelligence. Research in the
fields of decision making, leadership, and collective intelligence has been expanded and integrated, while the combined line of
leadership and decision making has been extended to the collective level of analysis. Overall, collective decision making was
viewed as a form of collective leadership, different from the leadership of a single, formal leader; and in both simulations and
a field study, the relationships between individual and collective intelligence and collective intelligence and collective decision
quality were supported. Despite the limitations of the current work, and with the promise of more rigorous future field research,
our results provide some support for the conceptual model. There is some alignment between the results of the agent-based
simulation and the field study and both offer researchers a first empirical glimpse into the decision making processes of a collective.
A foundation has thus been built for the study of collective-level decision making, which is at its core collective leadership,
and future research can further study leadership and decision making in various types of collectives.

References
Amason, A.C. (1996). Distinguishing the effects of functional and dysfunctional conflict on strategic decision making: resolving a paradox for top management teams.
Academy of Management Journal, 39(1), 123–148.
Anthony, W.P. (1978). Participative management. Reading. MA: Addison-Wesley.
Aristotle. (350 B. C. E). Politics (Vol. III). (B. Jowett, Trans.)
Arvey, R.D., Bouchard, T.J., Jr., Carroll, J.B., Cattell, R.B., Cohen, D.B., Dawis, R.V., et al. (1994, December 13). Mainstream science on intelligence. The Wall Street Journal,
A19.
Balkundi, P., & Harrison, D.A. (2006). Ties, leaders, and time in teams: Strong inference about network structure’s effects on teamviability and performance. Academy of
Management Journal, 49(1), 49–68.
Balkundi, P., & Kilduff, M. (2005). The ties that lead: A social network approach to leadership. The Leadership Quarterly, 17(4), 419–439.
Bantel, K.A., & Jackson, S.E. (1989). Top management and innovations in banking: Does the composition of the top team make a difference? Strategic Management
Journal, 10, 107–112.
Bass, B.M. (1949). An analysis of the leaderless group discussion. Journal of Applied Psychology, 33(6), 527–533.
Bass, B.M. (1954). The leaderless group discussion. Psychological Bulletin, 51(5), 465–492.
Bass, B.M. (2008). Handbook of leadership. New York: Free Press.
Bertucci, A., Johnson, D.W., Johnson, R.T., & Conte, S. (2011). The effects of task and resource interdependence on achievement and social support: An exploratory study
of Italian children. The Journal of Psychology, 145(4), 343–360.
Binder, K., & Heerman, D.W. (2002). Monte Carlo Simulations in Statistical Physics. An Introduction (4th ed.) Berlin: Springer.
Burke, C.S., DiazGranados, D., & Salas, E. (2011). Team leadership: A review and look ahead. In A. Bryman, D. Collinson, K. Grint, B. Jackson, & M. Uhl-Bien (Eds.), The
SAGE Handbook of Leadership (pp. 338–351). London: Sage.
Burke, C.S., Stagl, K.C., Klein, C., Goodwin, G.F., Salas, E., & Halpin, S.M. (2006). What type of leadership behaviors are functional in teams? A meta-analysis. The
Leadership Quarterly, 17, 288–307.
Caflisch, R.E. (1998). Monte Carlo and quasi-Monte Carlo methods. Acta Numerica, 7, 1–49.
Campbell, D.J. (1988). Task complexity: A review and analysis. Academy of Management Review, 13(1), 40–52.
Campbell, D.J., & Gringrich, K. (1986). The interactive effects of task complexity and participation on task performance: A field experiement. Organizational Behavior
and Human Decision Processes, 38, 162–180.
Campbell, J.P. (1990). The role of theory in industrial and organizational psychology. In M.D. Dunnette, & L.M. Hough (Eds.), (2nd ed.). Handbook of I/O Psychology, 1.
(pp. 39–74). Palo Alto, CA: Consulting Psychology Press.
Campion, M.A., Medsker, G.J., & Higgs, A.C. (1993). Relations between work group characteristics and effectiveness: Implications for designing effective work groups.
Personnel Psychology, 46, 823–850.
Carson, J., Tesluk, P., &Marrone, J. (2007). Shared leadership in teams: An investigation of antecedent conditions and performance. Academy ofManagement Journal, 50,
1217–1234.
Chi, M.T., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R.J. Sternberg (Ed.), Advances in the psychology of human intelligence. Hillsdale, NJ: Erlbaum.
Cummings, T. (1978). Self-regulated work groups: A sociotechnical synthesis. Academy of Management Review, 3, 625–634.
Dansereau, F., & Yammarino, F.J. (1998). A multiple-level leadership mosaic: One way to put the pieces together. In F. Dansereau, & F.J. Yammarino (Eds.), Leadership:
The multiple-level approaches (Part B) (pp. 327–349). Stamford, CT: JAI Press.
Dansereau, F., Alutto, J.A., & Yammarino, F.J. (1984). Theory testing in organizational behavior: The varient approach. Englewood Cliffs, NJ: Prentice-Hall.
Dansereau, F., Yammarino, F.J., & Kohles, J. (1999). Multiple levels of analysis from a longitudinal perspective: Some implications for theory building. Academy of
Management Review, 24, 346–357.
Day, D., Gronn, P., & Salas, E. (2004). Leadership capacity in teams. The Leadership Quarterly, 15, 857–880.
Dionne, S.D., & Dionne, P.J. (2008). Levels-based leadership and hierarchical group optimization: A simulation. The Leadership Quarterly, 19, 212–234.
Dionne, S.D., Chun, J.U., Hao, C., Serban, A., Yammarino, F.J., & Spangler, W.D. (2012). Article quality and publication impact via levels of analysis incorporation: An
illustration with transformational/charismatic leadership. The Leadership Quarterly, 23(6), 1012–1042.
Dionne, S.D., Gupta, A., Sotak, K.L., Shirreffs, K.A., Serban, A., Hao, C., et al. (2014). A 25-year perspective on levels of analysis in leadership research. The Leadership
Quarterly, 25, 6–35.
Dionne, S.D., Sayama, H., Hao, C., & Bush, B.J. (2010). The role of leadership in sharedmentalmodel convergence and teamperformance improvement: An agent-based
computational model. The Leadership Quarterly, 21(6), 1035–1049.
Durham, C.C., Knight, D., & Locke, E.A. (1997). Effects of leader role, team-set goal difficulty, efficacy, and tactics on team effectiveness. Organizational Behavior and
Human Decision Processes, 72(2), 203–231.
Epstein, J.M. (1999). Agent-based computational models and generative social science. Complexity, 4, 41–60.
Fleishman, E.A., & Mumford, M.D. (1989). Abilities as causes of individual diffrences in skill acquisition. Human Performance, 201–222.
Friedrich, T.L., Vessey,W.B., Schuelke,M.J., Ruark, G.A., &Mumford,M.D. (2009). A framework for understanding collective leadership: The selective utilization of leader
and team expertise within networks. The Leadership Quarterly, 20, 933–958.
Frost, C.H.,Wakely, J.H., & Ruh, R.A. (1974). The Scanlon Plan for organizational development: Identity, participation, and equity. East Lansing: Michigan State University
Press.
Frost, P., &Mahoney, T. (1976). Goal-setting and the task process: An interactive influence on individual performance. Organizational Behavior and Human Performance,
17, 328–350.
Gettinger, M., & White, M.A. (1979). Which is the stronger correlate of school learning? Time to learn or measure intelligence? Journal of Educational Psychology, 71,
405–412.
Gilliland, W.G., & Landis, R.S. (1992). Quality and quantity goals in a complex decision task: Strategies and outcomes. Journal of Applied Psychology, 77(5),
672–681.
Goldspink, C. (2002). Methodological implications of complex systems approaches to sociality: Simulation as a foundation for knowledge. Journal of Artificial Societies
and Social Simulation, 5(1), 1–19.
Guzzo, R.A., & Salas, E. (Eds.). (1995). Team decision making in organizations. San Francisco, CA: Jossey-Bass.
Hackman, R. (1969a). The nature of the task as a determiner of job behavior. Personnel Psychology, 22, 435–443.
Hackman, R. (1969b). Toward the understanding of the role of tasks in behavioral research. Acta Psychologica, 31, 97–128.
Herrnstein, R.J., & Murray, C. (1994). The bell curve: Intelligence and class structure in American life. New York: Free Press.
House, R.J., Rousseau, D.M., & Thomas-Hunt, M. (1995). The meso paradigm: A framework for the integration of micro and macro organizational behavior. In L.L.
Cummings, & B.M. Staw (Eds.), Research in organizational behavior, Vol. 17. (pp. 71–114). Greenwich, CT: JAI Press.
Howell, W.C., & Kreidler, D.L. (1963). Information processing undercontradictory instruction sets. Journal of Experimental Psychology, 65, 39–46.
Hrastinski, S. (2008). The potential of synchronous communication to enhance participation in online discussions: A case study of two e-learning courses. Information
& Management, 45, 499–506.
Iandoli, L., Klein,M., & Zollo, G. (2013). Can we exploit intelligence for collaborative deliberation? The case of the climate change collaboratorium. Cambridge, MA MIT
Sloan School Working Paper (pp. 4675–4708).
Keeney, R., & Raiffa, H. (1976). Decisions with multiple objectives: Preferences and value trade offs. New York, NY: Wiley.
Klein, K.J., Dansereau, F., & Hall, R.J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 105–229.
Koopman, P.L., & Wierdsma, A.F. (1998). Participative management. In P.J. Doentu, H. Thierry, & C.J. de-Wolf (Eds.), Personnel psychology: Handbook of work and
organizational psychology, Vol. 3. (pp. 297–324). Hove, U.K.: Psychology Press/Erlbaum.

Kozlowski, S.W., Chao, G.T., Grand, J.A., Braun, M.T., & Kuljanin, G. (2013). AdvancingMultilevel Research Design: Capturing the Dynamics of Emergence. Organizational
Research Methods, 16, 581–615.
Kozlowski, S.W., Gully, S.M., Salas, E., & Cannon-Bowers, J.A. (1996). Team leadership and development: Theory, principles, and guidelines for training leaders and
teams. Advances in interdisciplinary studies of work teams: Team leadership, 3, 253–291.
Kraiger, K., &Wenzel, L.H. (1997). A framework for understanding and measuring shared mental models of team performance and team effectiveness. In E. Salas, M.T.
Brannick, & C. Prince (Eds.), Team performance, assessment, and measurement: Theory, methods, and applications. Hillsdale, NJ: Erlbaum.
Langville, A.N., & Meyer, C.D. (2003). Deeper inside PageRank. Internet Mathematics, 1(3), 335–380.
Likert, R. (1977). The nature of highly effective groups. In J.L. Gray, & F.A. Starke (Eds.), Readings in organizational behavior (pp. 100–129). Columbus, OH: Merrill.
Macal, C.M., & North, M.J. (2010). Tutorial on agent-based modelling and simulation. Journal of Simulation, 4(3), 151–162.
MacCrimmon, K. (1976). An overview ofmultiple objective decision-making. In J. Cochrane, &M. Zelery (Eds.), Multiple criteria decision-making. Columbia, SC: University
of South Carolina Press.
Marion, R., & Uhl-Bien, M. (2001). Leadership in complex organizations. The Leadership Quarterly, 12, 389–418.
Mathieu, J.E., Marks, M.A., & Zaccaro, S.J. (2002). Multi-team systems. In N. Anderson, D.S. Ones, H.K. Sinangil, & C. Viswesvarian (Eds.), Handbook of Industrial, Work,
and Organizational Psychology. 2. (pp. 289–313). London: Sage.
Metropolis, N., Rosenbluth, A.W., Rosnebluth, M.N., Teller, A.H., & Teller, E. (1953). Equation of state calculations by fast computing machines. Journal of Chemical
Physics, 21, 1087–1092.
Miller, K.I., & Monge, P.R. (1986). Participation, satisfaction, and productivity: A meta-analytic review. Academy of Management Journal, 29(4), 727–753.
MIT Center for Collective Intelligence (2010, September 19). Main Page. Retrieved October 13, 2010, from Handbook of Collective Intelligence. http://scripts.mit.edu/
~cci/HCI/index.php?title=Main_Page
Mumford, M.D., & Connelly,M.S. (1991). Leaders as creators: Leader performance and problemsolving in ill-defined domains. The Leadership Quarterly, 2(4), 289–315.
Nonaka, I., & von Krogh, G. (2009). Tacit knowledge and knowledge conversion: Controversy and advancement in organizational knowledge creation theory.
Organizational Science, 20(3), 635–652.
Parry, K.,Mumford,M.D., Bower, I., & Watts, L.L. (2014). Qualitative and historiometricmethods in leadership research: A reviewof the first 25 years of The Leadership
Quarterly. The Leadership Quarterly, 25, 132–151.
Pearce, C.L., & Conger, J.A. (2003). Shared leadership: Reframing the hows and whys of leadership. Thousand Oaks, CA: Sage.
Pearce, C.L.,Manz, C.C., & Sims, H.P. (2008). The roles of vertical and shared leadership in the enactment of executive corruption: Implications for research and practice.
The Leadership Quarterly, 19, 353–359.
Roediger, H.L. (1990). Implicit memory: Retention without remembering. American Psychologist, 45(9), 1043–1056.
Rousseau, D.M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In L.L. Cummings, & B.M. Staw (Eds.), Research in
organizatioal behavior. 7. (pp. 1–37). Greenwich, CT: JAI Press.
Schmidt, F.L., & Hunter, J.E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research
findings. Psychological Bulletin, 124, 262–274.
Schriesheim, C.A., Castro, S.L., Zhou, X.T., & Yammarino, F.J. (2001). The folly of theorizing “A” but testing “B″: A selective level-of-analysis review of the field and a
detailed leader–member exchange illustration. The Leadership Quarterly, 12, 515–551.
Schroder, H., & Suedfeld, P. (1971). Personality theory and information processing. New York: Ronald Press.
Schroder, H., Driver, M., & Streufert, S. (1967). Human information processing. New York: Holt, Rinehart and Winston.
Scully, J.A., Kirkpatrick, S.A., & Locke, E.A. (1995). Locus of knowledge as a determinante of the effects of participation on performance, affect, and perceptions.
Organizational Behavior and Human Decision Processes, 61(3), 276–288.
Serban, A., Yammarino, F.J., Dionne, S.D., Kahai, S. S., Hao, C., McHugh, K.A., et al. (2015). Leadership emergence in face-to-face and virtual teams: A multi-level model
with agent-based simulations, quasi-experimental and experimental tests. The Leadership Quarterly, 26(3), 402–418.
Shuffler, M.L., Salas, E., Yammarino, F.J., Serban, A., & Shirreffs, K. (2012). Putting the “we” in leadership: Continuing the dialogue to advance our science and practice.
Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 440–446.
Snow, R.E., & Lohman, D.R. (1986). Toward a theory of cognitive aptitude for learning from instruction. Journal of Educational Psychology, 41, 1029–1031.
Somech, A. (2003). Relationships of participative leadership with relational demography variables: A multi-level perspective. Journal of Organizational Behavior, 24,
1003–1018.
Sosik, J.J., & Dinger, S.L. (2007). Relationships between leadership style and vision content: The moderating role of need for social approval, self-monitoring, and need
for social power. The Leadership Quarterly, 18, 134–153.
Spearman, C. (1904). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15(2), 201–292.
Stewart, G.L. (2006). A meta-analytic review of relationships between team design features and team performance. Journal of Management, 32, 29–55.
Surowiecki, J. (2004). The wisdom of crowds. New York: Doubleday.
Szuba, T.M. (2001). Computational collective intelligence. New York: John Wiley & Sons.
Thrash, T.M., & Elliot, A.J. (2003). Inspiration as a psychological construct. Journal of Personality and Social Psychology, 84(4), 871–889.
Uhl-Bien, M., & Marion, R. (2009). Complexity leadership in bureaucratic forms of organizing: A meso model. The Leadership Quarterly, 20, 631–650.
Uhl-Bien, M., Marion, R., & McKelvey, B. (2007). Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era. The Leadership
Quarterly, 18, 298–318.
Vancouver, J.B., Tamanini, K.B., & Yoder, R.J. (2010a). Using dynamic computationalmodels to reconnect theory and research: Socialization by the proactive newcomer
as example. Journal of Management, 36, 764–793.
Vancouver, J.B.,Weinhardt, J.M., & Schmidt, A.M. (2010b). A formal, computational theory ofmultiple-goal pursuit: Integrating goal-choice and goal-striving processes.
Journal of Applied Psychology, 95, 985–1008.
Vroom, V.H., & Deci, E.L. (Eds.). (1960). Management and motivation. Baltimore: Penguin Books.
Vroom, V.H., & Yetton, P.W. (1973). Leadership and decision making. Pittsburgh, PA: University of Pittsburgh Press.
Wonderlic Personnel Test, Inc. (1992). Wonderlic Personnel Test and Scholastic Level Exam: User’s Manual. Libertyville, IL: Wonderlic Personnel Test, Inc.
Woolley, A.W., Chabris, C.F., Pentland, A., Hashmi, N., & Malone, T.W. (2010). Evidence for a collective intelligence factor in the performance of human groups.
Sciencexpress, 330, 686–688.
Yammarino, F.J., & Dansereau, F. (2009). A new kind of organizational behavior. In F.J. Yammarino, & F. Dansereau (Eds.), Multi-level issues in organizational behavior
and leadership, Vol. 8. (pp. 13–60). Emerald Group Publishing Limited.
Yammarino, F.J., Dionne, S.D., Chun, J.U., & Dansereau, F. (2005). Leadership and levels of analysis: A state of the science review. The Leadership Quarterly, 16, 879–919.
Yammarino, F.J., Mumford, M.D., Vessey, W.B., Friedrich, T.L., Ruark, G.A., & Brunner, J.M. (2010a). Collective leadership measurement for the U.S. Army. Technical Report,
Contract No.W91WAW-09-C-0090. Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
Yammarino, F.J.,Mumford,M.D., Vessey,W.B., Friedrich, T.L., Ruark, G.A., & Brunner, J.M. (2010b). Collective leadership measurement for the U.S. Army: Measures. Research
Note, Contract No. W91WAW-09-C-0090. Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
Yammarino, F.J., Salas, E., Serban, A., Shirreffs, K., & Shuffler, M.L. (2012). Collectivistic leadership approaches: Putting the “we” in leadership science and practice.
Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 382–402.
Yukl, G. (2009). Leadership in organizations. Upper Saddle River, NJ: Prentice Hall.