Behavioral Ecology Advance Access originally published online on March 3, 2007
Behavioral Ecology 2007 18(3):541-550; doi:10.1093/beheco/arm005
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Choice processes in multialternative decision making
Zoology Department, University of Oxford, South Parks Road, OX1 3PS Oxford, UK
Address correspondence to A. Kacelnik. E-mail: alex.kacelnik{at}zoo.ox.ac.uk.
Cynthia Schuck-Paim is now at the Department of Experimental Psychology, University of Sao Paulo, Rua Tucuma 141—A. 102, 01455-010 Sao Paulo, SP, Brazil. E-mail: cynthia{at}origem.info.
Received 19 July 2006; revised 28 January 2007; accepted 29 January 2007.
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
We study how the mechanisms of choice influence preferences when animals face more than 2 alternatives simultaneously. Choice mechanisms can be hierarchical (if alternatives are assigned to categories by their similarity and choice is between categories) or simultaneous (if options enter the choice process individually, each with its own value). The latter, although simpler, can lead to counterintuitive outcomes because expressed preference between options depends not only on the kinds of options present but also on the number of exemplars within each kind, so that decision makers have a higher probability of picking an option of a given class when exemplars in this class are common. Higher preference for commoner options has indeed been shown in humans, and if present in animals, it would affect many choice domains, including prey and mate choice. We studied the problem using starlings making risk-sensitive choices. Subjects chose between a risky option and 1 (in binary choices) or 2 (in trinary choices) fixed options that were identifiable as distinct but were identical in reward rate and had no variance. Preference between the risky and each fixed option was unaltered between binary and trinary contexts, but subjects chose a higher proportion of the fixed kind when this was represented by 2 rather than 1 distinct food sources. This means subjects were objectively risk prone in binary and risk averse in trinary contexts. These results fit accounts based on learning principles, but contradict the expectations of functional models of choice, including risk-sensitivity theory.
Key words: choice, decision making, foraging, learning, rationality, risk sensitivity.
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
We study the process by which animals make choices in multialternative situations. Our experimental system is based on risk sensitivity (Stephens 1981
), but we intend to address the general problems raised in decision making when more than 2 options are simultaneously available.
To illustrate the problem, we consider an animal that can choose between foraging options differing in their level of variance in reward rate, as shown in Figure 1. The figure shows 3 symbols (stimuli) on the left (R, F1, and F2) representing options open to the decision maker. As we have a European starling (Sturnus vulgaris) in mind when illustrating the problem, these symbols could stand for species of insect prey identified by their color, foraging sites, or modes of foraging such as hawking or plucking plants to seek grubs in the hollows. In the example, the outcome of stimulus R is stochastic: it is sometimes a big and sometimes a small reward, each with different fitness consequences. The expected fitness consequence of choosing the risky option R (WR) takes into account this variability (McNamara and Houston 1992). The outcomes of F1 and F2 are fixed: each yields always the same reward. In this case, we use an example where the 2 outcomes yield equal fitness (WF) to emphasize that they can be integrated into a single category, but the question on which we focus does not need this simplification. For instance, several low variance options could be considered into a single (low-risk) category even if they had small differences in yield.
|
The task we face is to predict and explain, both functionally and mechanistically, choices among such a set of alternatives. Conventionally, the optimal proportion of choices between any 2 stimuli would be predicted considering their fitness consequences: the subject should be indifferent between F1 and F2 and have a certain preference between R and either of the fixed options (Kacelnik and Bateson 1996
). We denote the predicted preference for the risky option by Q and indicate the choice set with parentheses. Thus, Q(WR, WF) is the predicted preference for the risky option R as computed using the expected fitness consequences of R and F, as is the norm in optimality modeling. We restrict our discussion to cases where 0 < Q < 1, namely, excluding absolute preferences for only one option, because partial preferences are an almost universal empirical regularity in repeated choice experimental studies.
The problem sets off as soon as we consider choices between more than 2 options. Binary choices dominate the literature (Real et al. 1982; Wunderle and O'Brien 1985; Caraco et al. 1990; Cartar and Dill 1990; Reboreda and Kacelnik 1991; Brito e Abreu 1998), not because of greater relevance but because they are simpler to model and test, and perhaps because until now it was implicitly assumed that adding more exemplars of one of the options would not alter the results. Trinary choices, however, are important because they pose a host of new and fundamental problems that need to be addressed to generalize experimental results to natural circumstances. To model decision making in trinary situations and in the presence of partial (as opposed to exclusive) levels of preference (arguably more relevant to natural choices), we consider 2 contrasting processes which we shall call "hierarchical" or "simultaneous."
In a hierarchical process, subjects' first group options depending on their degree of similarity (in our example, there would be 2 categories: "fixed" and "risky"). They then determine relative preference between the 2 categories and distribute their choices indifferently among options of equal value within each category. In the example, the decision maker builds up a preference ranking between WR and WF, assigning Q choices to R and 1 – Q to the rest. A risk neutral (rate maximizer) subject without the constraint of partial preferences would thus compute the mean rate of gain for each option and take exclusively the prey with highest expectation. If instead the subject is risk sensitive, it should choose among the risky option and the best of the 2 fixed alternatives. If the 2 fixed options yield equal fitness, it should choose among the risky one and either of the fixed ones, with indifference between the 2, so that preferences would average (1 – Q)/2 for each fixed option. Thus, the total proportion of choices for the categories risky and fixed is the same regardless of whether there is one or more fixed alternatives, but each single fixed option receives less choices when part of a trio. The process just described has been mostly studied in the context of human choices and is known as "hierarchical elimination model" or "similarity hypothesis" (Tversky and Russo 1969; Tversky 1972, but see also Shafir et al. 2002 for an example with nonhumans animals).
In the simultaneous process, the subject computes the value of food sources and then distributes its choices among all alternatives by using some algorithm that treats individual options rather than categories as the units. So in the example, the subject expresses preferences among R, F1, and F2 considered separately. We call the proportion of choices for any alternative under this process S and denote the proportion of risky choices by S(R, Fi) in binary choices between R and one of the fixed options and by S(R, F1, F2) in the trinary choices. The significant issue here is that while the preference for the risky category (Q) based on the hierarchical process is the same in the binary and trinary situations, "S" differs between the 2 choice contexts. For example, if the likelihood of selecting any option under the simultaneous process of choice is some growing function of its value relative to the value of all other options (with "value" given by the learned consequences associated with each option), then overall preference for R would decrease as the number of fixed options increases, as each new fixed option will take a share of choices from all other options (including R). This sensitivity to the number of least preferred alternatives is, however, not predicted by behavioral ecological models of optimal choice (Charnov 1976).
It is unlikely that only one of these 2 choice processes will exist in nature, and each situation may need to be examined. For example, in some cases, stimuli cueing options with similar outcomes will be also very similar, especially if there is some coincidence between the kinds of prey species and their properties. If, say, beetles are a risky option (a fraction of them is toxic) and there are also 2 species of butterflies (each leading to the same fixed fitness outcome), classifying the 2 fixed options as being the same would be easy, and modeling the process hierarchically as choosing first between the categories risky and fixed (which in this case correspond to choosing between "beetle" and "butterfly") and later between the elements within each category (namely, between the 2 butterfly species) would lead to accurate predictions. However, this need not be the most common situation. It is possible that, in addition to beetles and butterflies, there are also leatherjackets available, which have the same fixed fitness value as butterflies, but look very different. In this case, it is important to know if choices for the risky option (beetles) would differ depending on the number of prey species yielding an equivalent fixed outcome. In other words, would a choice between beetles and butterflies lead to more beetles being consumed than a choice between beetles, butterflies, and leatherjackets? If so, preferences would not be predictable on the basis of risk-sensitivity analysis.
Although it would be natural for behavioral ecologists to favor the hierarchical mechanism, the prevalent analysis of this problem within the economic psychology literature has considered both approaches, and if anything, formal analyses favor the latter. For example, the simultaneous handling of choices is implied by the so-called principle of independence from irrelevant alternatives (IIA), a recurrent principle of economic theories of rational choice that guarantees the existence of some scale of value (e.g., monetary gains, satisfaction, well-being) governing choices. IIA is in fact best captured in the choice axiom of Luce (1959) or "constant-ratio rule" (Clarke 1957). This is a restrictive principle used to test internal coherence in choices and states that the inclusion of a new option into a choice set should take from preexisting options in proportion to their original shares (as required if choices are made simultaneously), hence relative choice proportions between any pair of options should always be the same regardless of the number of options in the set.
Breaches of the choice axiom (namely, cases where the addition of a third choice changes the relative distribution of choices between the 2 previous options) have however been frequently reported in the psychological and economic literature (Huber et al. 1982; Wedell 1991; Payne et al. 1992; Simonson and Tversky 1992; Tversky and Simonson 1993; Slaughter et al. 1999). For example, in what has become known as the "attraction effect," the addition of a relatively inferior alternative (or decoy) into an existing choice set has been shown to increase the proportion of choices for an relatively superior option from the original set, thereby in violation of the choice axiom (for a review of decoy types, see Wedell and Pettibone 1996). Similarly, a number of studies have shown that the relative choice probability of an option will increase when it becomes a "compromise" (say, between price and quality) in a choice set (a phenomenon known as the "compromise effect," Simonson 1989). More recently, violation of this sort was also found in some pioneering studies with animals (but see Schuck-Paim et al. 2004). For instance, studying the foraging preferences of honeybees (Apis mellifera) and gray jays (Perisoreus canadensis), Shafir et al. (2002) showed that the relative preference of the bees and jays between 2 original foraging options changed with the introduction of an additional, unattractive alternative. Similarly, Bateson et al. (2002) analyzed the preferences of wild rufous hummingbirds (Selasphorus rufus) for artificial flowers, showing that the relative preferences of the birds for one of the 2 target options was higher in a trinary context (where a third option, or ‘decoy’, was available) than in binary choices (only 2 options on offer). The same was true in a study designed to test the risk-sensitive foraging preferences of European starlings (S. vulgaris) for variability in amount of food (Bateson 2002): the relative preferences of the starlings for the less variable options were always higher in a trinary than in a binary context. Such breaches have been generally interpreted as providing evidence for context-dependent valuation, namely, the notion that preference for an option would be constructed at the moment of choice depending on the presence and properties of other alternatives.
Most work on animal choice has addressed pairwise choices, but real life scenarios may involve multiple alternatives. Because in these cases, preferences differ depending on the choice process employed by the animal, it is possible that many results obtained to date, as well as the theoretical concepts tested with such data, do not hold in more complex scenarios (Bateson and Healy 2005). Our goal here is to distinguish experimentally the 2 alternative processes and to examine the significance of this issue for functional modeling. We explore binary and trinary choices using European starlings (S. vulgaris) as subjects, and although we structure our experiments in terms of options that differ in the variance of rewards, we are concerned with the implications of hierarchical and simultaneous mechanisms in broader choice domains.
|
METHODOLOGY |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
Experimental design
We compared starlings' preferences between binary and trinary choice contexts, examining how the inclusion of a third food source changed the relative level of preference between the 2 original options. The third food source had the same properties as one of the members of the original pair.
In each treatment, subjects were exposed to 3 alternatives, 2 of which shared exactly the same parameter values, as shown by the 2 fixed options in the example of Figure 1. The decision problem was therefore designed to distinguish the processes described in the introduction: categorical, where the subject groups alternatives according to similarity in their properties and then chooses between categories or simultaneous, where, it assigns a value to each individual option and then chooses among all options present.
The sources of reward were characterized by the mean and variance of the delay that subjects had to wait to receive food: the 2 equivalent options offered a same fixed delay to food, whereas the third offered a variable delay with 2 equiprobable outcomes. There were 5 treatments, which differed in the delay offered by the 2 fixed options (ranging from a relatively short—as compared with the mean delay offered by the variable option—to a relatively long delay). Previous experimental results on risk sensitivity show that starlings (and other animals) in these scenarios favor the variable delay alternative but that preferences are partial (as opposed to exclusive) (Bateson and Kacelnik 1995, 1998; Brito e Abreu 1998). On these basis, we expect starlings to show a preference for the fixed delay when it (the fixed delay) is much shorter than the average of the delays in the variable option and, conversely, to show preference for the variable delay when this average is equal or shorter than the delay offered by the fixed options. The point of indifference when only 2 options (one fixed and one variable) are present is, on the basis of both theoretical interpretations and empirical findings, expected to be when the fixed option lies in the region around the harmonic or geometric mean of the 2 delays of the variable option (Gibbon et al. 1988; Bateson and Kacelnik 1996). By varying the delay of the fixed options among treatments, we can thereby analyze the fit between the observed levels of preference for the options and the predictions of both mechanistic hypotheses along a wide range of partial levels of preference.
Subjects
Subjects were 10 wild-caught European starlings, captured in Oxford, UK (English Nature licence 20020068) as adults on March 2001. Prior to the experiments, the starlings were kept in 2 outdoor aviaries (3 m high x 3.2 m wide x 4 m long) and fed ad libitum on a mixture of turkey crumbs, Orlux pellets, and mealworms. Fresh drinking and bathing water was always available. Five days before the beginning of the training sessions, the birds were transferred to the laboratory and housed in individual cages (120 cm long x 50 cm wide x 60 cm high) that served both as home cages and experimental chambers. During this period of adaptation, all birds had free access to water and ad libitum food. During the experimental period, the starlings were given free access to food (turkey starter crumbs) after the end of each daily session, from 1700 h until 1900 h. At 1900 h, they were supplemented with 10 mealworms, and then food was removed. From previous experience, this regime is known to allow the starlings' body weights to remain stable at approximately 90% of their free feeding value or above (Bateson 1993). Fresh drinking water was always available, and bathing trays were provided daily. There was no mortality or any observed adverse effects on any of the subjects. The experiments were initiated in November 2001 and completed in March 2002. All subjects were released back into the wild during the following spring.
Apparatus
The experimental cages had a panel with a food hopper, 3 response keys (at either side of the hopper and above it) and 2 perches. An Acorn Risk PC 600 computer running Arachnid software (Paul Fray Ltd, Cambridge, UK) controlled the contingencies and collected the data. During the experimental sessions, food rewards were 2 units of Noyes precision pellets (0.02 g each), delivered at a rate of 1 unit/s. Lights in the experimental rooms were gradually switched on at 0530 h and off at 1930 h, and temperature ranged from 11 °C to 15 °C. Subjects were visually but not acoustically isolated.
Experimental protocol
Subjects were first trained by a standard autoshaping procedure to peck at the keys to obtain rewards. Training stopped when all birds pecked in at least 80% of the trials.
All subjects experienced 5 treatments. In each treatment, the subjects experienced 3 options differing in the parameters of delay to reward: a variable option, offering equiprobable delays of either 12 or 28 s (coefficient of variation = 40%, Harmonic mean 16.8 s, geometric mean 18.3 s, arithmetic mean 20 s) and 2 fixed options both offering the same delay to reward. In treatments T12, T15, T18, T22, and T28, the delay offered by the fixed options was 12, 15, 18, 22 and 28 s, respectively. The variable option was the same in all treatments. The order of treatments was balanced across subjects following a balanced Latin-square design. As we had an odd number of treatments (n = 5), we used 2 separate arrangements comprising 5 orders each (Namboodiri 1972). Each of the options was associated with one color illuminating the pecking keys (green, red, and yellow). The association of colors with options was balanced across birds, but for a given bird, one color was always associated with one particular option. Although the subjects were already familiar with the experimental apparatus, they had not experienced colors as discriminative stimuli before.
A discrete trials procedure with a fixed intertrial interval (ITI) of 60 s was employed. There were 3 types of trials: no-choice trials, choice trials, and probe trials. No-choice trials were intended to provide the birds with information about each alternative and started with one of the keys blinking (0.7 s on, 0.3 s off). The first peck on the key caused the light to stay ON for the delay corresponding to the option presented, during which pecks on the key were counted. The first peck after the delay timed out triggered the delivery of the standard food reward (2 precision pellets) in the hopper, and a new ITI started. There were 4 kinds of no-choice trials, F1, F2, and V12 and V28, according to whether the trial was programmed as either of the 2 fixed options or either of the 2 values of the variable option, respectively.
Choice trials began with either 2 or 3 pecking keys (depending on whether the choice was binary or trinary) blinking. The first peck on any of the keys caused the chosen key to stay continuously ON for its respective delay and the remaining key lights to turn OFF. The first peck after the delay had elapsed triggered the delivery of the standard reward and the beginning of a new ITI. Probe trials were used to measure the subjects' knowledge of the temporal properties of each option and were identical to no-choice trials, with the only difference that the light stayed on for a delay 3 times longer than the delay programmed for that option and that no reward was given. There were 3 kinds of probe trials, PF1, PF2, and PV, according to whether the color on the key was that of either of the 2 fixed options or that of the variable option, respectively.
Subjects experienced 4 sessions per day, starting at 0600, 0900, 1200, and 1500 h. Each session comprised 5 blocks of 15 trials each and finished with an additional probe trial (total trials per session = 76, see description next). Each block started with one probe trial, where one of the 3 options was presented (presentation was random with the restriction that each option had to be presented twice per session in the probe trials). The probe trial was followed by 3 no-choice trials, one corresponding to each option, and then by 4 choice trials: 3 binary choice trials (one of each possible pairwise combination) and one trinary choice trial (all options simultaneously available). Choice trials were followed by another series of 3 no-choice trials and 4 choice trials, with the only difference that in the second series of no-choice trials, we programmed the variable option to offer the outcome that had not been experienced in the previous series. The order of presentation of the options was always randomly determined. Options were also not consistently associated with a particular side, but this was randomized between trials. Table 1 shows an example of a typical block.
|
The 2 morning sessions and the 2 afternoon sessions were each grouped for the purpose of analysis. For each bird, each treatment ended when the regression coefficient of the choice proportions of 6 consecutive sessions (against session number) was not significantly different from zero, and the standard deviation of the choice percentages of these 6 sessions did not exceed 10. This criterion had to be reached in at least 2 out of the 4 types of choice trials (3 binary and 1 trinary) for the treatment to end. The birds were given a minimum of 8 and maximum of 20 sessions per treatment. Data from the last 6 sessions were used for analysis. Subjects were given 3 resting days with ad libitum food between treatments.
|
DATA ANALYSIS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
Preference scores for the fixed options
In all treatments, the 2 fixed options offered the same delay to reward but differed in the color of the discriminative label associated to each of them. Accordingly, mean preference levels between the 2 fixed options were close to indifference (50%), but most individual subjects showed small deviations from 50% (not all toward the same colors). These preferences, although not significant, were associated with the color of the discriminative label associated to each option. Therefore, instead of using an arbitrary classification of the fixed options for the purpose of analysis (e.g., F1 and F2), the 2 fixed alternatives were classified for each subject and treatment in 2 new categories, as preferred fixed option (FP) and nonpreferred fixed option (FN).
Relative preferences
The 2 possible choice mechanisms investigated here can be distinguished by their predictions regarding the differences in the relative level of preferences for the options between the binary and trinary contexts. To compare the subjects' relative preferences for each choice pair between contexts, we transformed choice proportions in the trinary situation by considering only 2 options at a time, so that the percentage of choices for the pair considered added up to 100%. For instance, preference for the preferred fixed option as compared with the variable option in the trinary situation—p(FP, V; {FP, FN, V})—is
 | (1) |
If the birds employ a hierarchical process of choice, grouping similar alternatives within the same category (categories fixed and variable in the present case) according to their consequences, the fixed options (FP and FN) should compete more with each other in terms of choice proportions than with the variable option (i.e., total choices for the category fixed should be split between FP and FN). Therefore, relative levels of preferences for each fixed option against the variable should be lower in the trinary than in the binary context. Specifically, if in a binary choice, the preference for the fixed option is FBi = p(F; {F, V}), then in the trinary choice, preference for each fixed option should on average be FBi/2. Relative preferences for any of the fixed options over the variable in the trinary situation should thus on average be (FBi/2)/(1 – FBi/2). This is to be expected from functional theories of risk sensitivity: if the subject aims at a certain level of risk, it should allocate some fraction of its behavior to the variable option and then distribute the rest among options of identical (or similar) characteristics and no variance.
Conversely, if the birds employ a simultaneous process of choice, the inclusion of a new option in the trinary situation should take a share of choice higher than FBi/2, and the overall level of preference for the variable option should decrease. In the specific case that each new option takes a share of choices proportional to its value (as given by the learned consequences of choosing the option), relative preferences between each pair of options should not change between the binary and trinary contexts. To see this with an example, suppose that in 60% of the choice opportunities, subjects choose the fixed option over the variable in the binary context (i.e., it is 1.5 times more likely to be chosen). If, in the trinary context, a new (identical) fixed option takes a share of choices proportional to its value, it should be also 1.5 times more likely to be chosen than the variable option (and equally likely to be chosen as the original fixed option). This means that this new fixed option would be chosen on approximately 37.5% of the choice opportunities, the original fixed option would be also chosen on 37.5% of the opportunities, and the variable option on 25%, and therefore, relative level of preference between any of the fixed options and the variable option remains unchanged (any fixed option is still 1.5 times more likely to be chosen), but the subject is twice as likely to choose fixed over variable in trinary respect to binary choices (namely, 70/2.5 = 3, as opposed to 60/40 = 1.5). From the perspective of risk sensitivity, this would be an objective shift to stronger risk aversion, even if no change in the state of the subject has occurred between the 2 contexts. In terms of predictions, we should then expect for this specific case the slope and intercept of the regression of the relative proportion of choices for any given pair of options (as opposed to any given category of options) in a binary situation against the relative proportion of choices for this same pair in a trinary situation (as calculated by Equation 1) not to be different from one and zero, respectively.
To test these predictions, we analyzed the relationship between relative preferences in the binary versus trinary contexts. In all analyses, we considered the relative proportion of choices in the trinary and binary situations to be the dependent and independent variables, respectively (choice proportions were square root arcsine transformed). To take into account the fact that each bird contributed with 5 data points to the regression (one corresponding to each treatment), we calculated the slope and intercept of the regression for each individual bird separately, subsequently testing whether the groups of 10 intercepts and 10 slopes (one intercept and one slope corresponding to each subject) were significantly different from zero and one, respectively. Because 2 possible pairs of options (FP x V and FN x V) were tested for the detection of the same phenomenon, we corrected significance levels by employing Bonferroni corrections for multiple comparisons.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
Knowledge about the alternatives
In Figure 2, we can see that the birds viewed the payoff from the 2 fixed alternatives as identical. The figure shows mean pecking rates for the fixed options during the delays experienced in the probe trials, with the number of pecks computed in 1 s time bins. It is clear from the figure that for both options (FP and FN), the response functions peaked at the appropriate times, namely, when food was due, in all treatments.
|
Figure 3 shows the mean pecking rate for the variable option in the 5 treatments. The magnitude of the pecking rates for the variable option progressively increases as the delay of the fixed options increases (i.e., as the variable option becomes "better" relative to the fixed option), but the shape of the function is maintained across treatments (remember that this option did not vary across treatments). Similarly to the results reported by Schuck-Paim and Kacelnik (2002)
, there were no discernible peaks at 12 and 28 s, the time bins when food was programmed. This can be interpreted as a fusion between the 2 peaks and is consistent with the scalar property of behavioral timing: because the spread of the response distribution is proportional to the length of the interval being timed, the presence of distinct peaks of responding is only noticeable for sufficiently large ratios between the 2 programmed intervals. In the present experiments, the ratio between the 2 delays corresponding to the variable option (12 and 28 s) was in the same range where no discernible peaks were observed by other authors (Catania and Reynolds 1968; Leak and Gibbon 1995). Taken together, these results strongly indicate that subjects were able to correctly time the delays associated with the options and that their timing accuracy varied in accordance with the scalar property.
|
Relative choice proportions
Figure 4 shows a plot of relative choice proportions between each fixed option and the variable option in the trinary context (calculated following Equation 1 in Data analysis) versus the proportion for the same choice in the binary context (namely, the number of choices for the option divided by the total number of choices). For the choice pair (FP, V), the group of individual slopes (hereafter s, see Data analysis) and intercepts (c) was not significantly different from one and zero, respectively (s = 0.90 ± 0.21; t9 = –1.43, P = NS; c = 0.039 ± 0.16; t9 = 0.75, P = NS). The same was observed for the choice pair (FN, V), with the intercept of the group of regressions not being different from zero (c = 0.03 ± 0.12; t9 = 0.86, P = NS) and the slopes not being different from one (s = 0.92 ± 0.21; t9 = –1.25, P = NS). The mean relative proportion of choices for each fixed option (average for all birds) in the binary context was also highly correlated with that in the trinary context (r
= 0.98 for the choice pair [FP, V] and r= 0.99 for the choice pair [FN, V]). These very high r2 values show therefore an extremely close fit to the predictions of the simultaneous model of choice.
|
Figure 5 shows the mean relative proportion of choices for each fixed option in the binary and trinary contexts in each of the treatments, together with the predictions of both processes. Similarly to the results reported above, the figure shows that, with the exception of one treatment in one of the choice scenarios, the observed preferences are extremely close to the prediction of choices based on a simultaneous process of choice but far from the predictions of the hierarchical model of choice.
|
To further investigate the observed results, we conducted a repeated-measures analysis of variance for each of the choice pairs having "treatment" (the value of the fixed delays) and "context" (binary or trinary choices) as within-subject factors and (transformed) choice proportions (calculated following Equation 1) as the dependent variable. As expected, for the choice pair involving the 2 fixed options (FP and FN), there was no effect of either treatment or context (F4,28 = 0.98, P = NS and F1,7 = 2.58, P = NS, respectively) on preferences. For the choice pairs (FP, V) and (FN, V), there was, as expected, a strong effect of treatment (F4,28 = 42.1, P < 0.001 and F4,28 = 47.6, P < 0.001, respectively), showing that the level of preference for the variable option decreased as the delay of the fixed options became shorter (i.e., as the fixed options became better), as shown in Figure 6. However, also as predicted, there was no effect of context (F1,7 = 0.90, P = NS and F1,7 = 0.6, P = NS, respectively), namely, no differences in relative choice proportions between the binary and trinary situations. For none of the choice pairs was the interaction between treatment and context significant (F4,28 = 1.44, P = NS and F4,28 = 1.18, P = NS, respectively). Altogether, these results show that relative preferences between any pair of options were not different between the binary and trinary contexts, in agreement with the predictions of simultaneous choice processes. Additionally, they were highly correlated (as indicated by the high r2 values shown).
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
Our results showed that the starlings did not choose between foraging options hierarchically. Even though subjects were able to identify that the outcomes of the fixed options were equal (they correctly tracked the options' temporal properties), they did not show stable preferences between the categories fixed and variable. Instead, they showed stable preferences between the variable option and each of the fixed alternatives, as if each fixed option was considered separately in the trinary situation, taking a share of choices from all other options. This result shows that the presence of a new option in a set can bias preferences in ways not foreseen by functional models of risky choice, as we discuss below. It is important to remember that our analysis, in common with most work in this field, takes as a fact that preferences between options are partial, and not absolute, as might be expected from functional analyses. We cannot tell why (in an adaptive sense) this should be so, but accept it as an almost universal property of choice behavior and proceed from there.
Our results bear important implications for the mechanistic and functional understanding of animal choices in multialternative scenarios. Amongst these, perhaps the most unanticipated is the observation that the use of a process of choice whereby every option is considered individually can lead to counterintuitive preference patterns such as preference reversals. To see this, suppose that in one of the choice situations elections for the variable option represented the optimal (or preferred) solution, with birds showing a probabilistic bias toward it in binary choices (the argument is equally valid for whichever choice is considered optimal). Now let the choice set include more fixed options, all of which suboptimal relative to the variable option. If subjects consider all options individually, each new fixed option included in the set should take its share of choices from all previously existing alternatives, including the variable option. As a consequence, the proportion of choices for the variable alternative should gradually drop with the inclusion of new fixed options, to the point that the overall probability of choosing it is reversed, becoming lower than that of choosing the category fixed. Indeed, such a preference reversal was observed here. Figure 6 shows that preferences were reversed between the binary and trinary contexts (PR, for preference reversal, Figure 6) in one of the treatments (T18): although starlings were risk prone in the binary situation, they were risk averse in the trinary, as most of their choices were taken by the 2 fixed alternatives. This treatment lies at the point where the fixed delay is close to the geometric and harmonic means of the 2 delays in the variable option, thus being the point where on the basis of previous experience and various mechanistic theories (including Expectation of the Ratios: Bateson and Kacelnik 1996; Scalar Utility Theory: Kacelnik and Brito e Abreu 1998 and Hyperbolic discounting: Mazur 1984) one would normally expect indifference and where behavior is maximally sensitive. The effect, however, is present throughout the range of values of the fixed delay, indicating that the phenomenon is not restricted to the conditions when the animal is indifferent, and choice is treated by the subject as inconsequential. Thus, the overall allocation of choices between the 2 categories (fixed and variable) changed with the size of the choice set. This effect should be even stronger with larger choice sets, to the point of producing higher levels of preference for a suboptimal or less-preferred category provided it contains many options.
Our main finding is that choices do not always reflect a process whereby preferences are predicted exclusively in terms of the options' payoffs. The new, third alternative, took a share of choices from all preexisting alternatives even when its payoff was inferior. Still, more than "stealing" some choices from all options, the new fixed alternative was chosen proportionally to its attractiveness in a binary situation. This pattern of preference is consistent with the constant-ratio rule (Luce 1959) as explained in the Introduction. Although the expression of partial levels of preference does not by itself maximize value (as an immediate maximizer would allocate all behavior to the option with highest value), the expression of partial preferences consistently with the constant-ratio rule has been recently used as a measure of internal consistency and coherence in choices in animal studies (choices would be "economically rational," Bateson 2002; Bateson et al. 2002, 2003; Shafir et al. 2002). Our findings are in this sense surprising as they show that the constant-ratio rule can also lead to preference reversals between "payoff categories," hence to specific violations of rationality predictions.
To understand the patterns of preference observed, one must look at the processes underlying the expression of nonexclusive choices for one option. An assumption of functional models of choice is that behavior maximizes the expected value of a given function, this function normally being fitness or some proxy versus behavior. Under stable scenarios, strict maximization requires exclusive choice of the best option, an expectation contradicted by the pervasive empirical presence of partial preferences. Nonexclusive choices in stable situations are often—and plausibly—interpreted as the outcome of processes whereby the choice probability of each option is a continuous function of its value. One possibility is that nonexclusive choice evolved to approximate adaptive responses to the need to track environmental changing opportunities, a phenomenon referred to as "sampling" (Krebs et al. 1978; Dow and Lea 1987). However, for sampling to account for the present preference patterns, it would be necessary that the frequency with which a subject sampled the environment (choosing less-than-best options) increased proportionally as the size of the choice set increased, to the point where the overall sampling frequency was higher than that of choosing the preferred alternative (as shown by the preference reversal illustrated in Figure 6). We consider this an unlikely possibility because the frequency with which an animal can sample its environment should be constrained by the need to reach a minimum energetic threshold enabling survival and reproduction. In other words, the greater the number of alternatives the greater would be the proportion of behavior used for sampling, and the subject would never focus on favorable options. A further reason to question interpretations based on sampling is that sampling should not be very prevalent where animals are faced with designs such as the present one, which included many no-choice trials to decrease the uncertainty of the subjects as to the programmed characteristics of each option.
Another argument relates to the degree of generality of the process uncovered by our experiment. It could be argued that the use of a simultaneous mechanism is restricted to cases where, as in our experiment, options with identical properties were signaled by different stimuli. If cues encountered in nature are reliable signals of an option's properties, then options sharing similar properties would be typically identified by similar cues. For instance, if options are potential mates that offer honest signals of quality, candidates of the same quality should give the same signal and would thus be easier to categorize together. Thus, the association here of each fixed option to a different color could have led the starlings to allocate their choices between them as if they were distinct alternatives, even though the starlings did identify the outcomes as having the same temporal parameters. The implication in this argument is that sensorial cues are what is used for categorization or grouping of the options, rather than payoffs. We believe that probably both situations can happen in nature and are logically viable. Consider, for example, the decisions of a central place forager. At the time the animal has to decide, for instance, to which field it will go, it remembers that the northern field is on average more profitable, but the eastern and southern fields sometimes offer larger rewards. Say that given its energetic state, the forager decides to avoid the risks of the eastern and southern fields and forage instead in the security of the northern one. In this case, the fields are all signaled by location and landmarks that bear no intrinsic information about their richness or variability, hence the bird makes its choices using the memory of their outcomes. Thus, categorization by payoffs should be expected if the predictions of risk-sensitive foraging theory are to be accomplished—an expectation contradicted by our findings.
Additionally, evidence from studies on frequency-dependent food selection do not support the hypothesis that, had we associated the 2 fixed options to the same colors, the starlings would have chosen them as if belonging to the same payoff category. For example, in a typical study an animal is given a choice between 2 food types, presented in different relative frequencies (e.g., a bumblebee is presented with an array of artificial flowers of 2 colors, where each color is available in different frequencies in different treatments; Smithson and MacNair 1996). The experimental conditions are thus analogous to those employed here, with the only difference that options with identical outcomes are signaled by the same cues. Similarly to what we found, in these studies animals eat disproportionally more of the common food item (a phenomenon termed "proapostatic selection"). Although preferences are not always as extreme as those found here (with a new alternative taking choices away from all previous options in proportion to its attractiveness), the observation that the more frequent a food type the higher its consumption (Allen 1988) indicates that even when similar outcomes are cued by similar stimuli, animals still fail to include them in the same payoff category when choosing.
From a mechanistic perspective, our results might be understood by considering the processes whereby subjects learn about the options. Here (an indeed in most experimental analyses of behavior), subjects did not choose directly among the outcomes of the options. Instead, they learned to choose among different stimuli. One possibility is that preference for these stimuli depended on how reliable they consider the color of each option as a predictor of the corresponding reward, namely, how strong the association between the color and its outcome was (technically, preferences would depend on the option's "associative strength"; see also Montague et al. 1995; Kacelnik and Bateson 1996; Shapiro 2000; Shapiro et al. 2001, for a possible link between associative learning and risk sensitivity). This mechanism does not assume the grouping of options into categories of similar payoffs. On the contrary, if the likelihood of selecting an option is some function of its associative strength relative to the associative strength of other options, then preferences in choice contexts involving 2, 3, or more options should lead to the preference patterns observed here. At present, the main weakness of this approach is that little is known about the rules that translate associative strength into choice. In the few cases where a decision rule was fitted to the data, generalized matching (Couvillon and Bitterman 1987; Shapiro 2000)—a rule entirely compatible with the constant-ratio rule but lacking functional justification—was predominantly favored. Yet, the general compatibility of our results with the predictions of learning theory indicates that the use of a choice process whereby each option is handled individually and not in categories may have derived from learning processes that evolved in situations of greater overall importance than the potential fitness loss they could cause. Because this appears to be, however, the first study to address directly this issue, further studies are needed to test the generality of the phenomenon reported here.
The implementation of variations of our design so as to include other choice domains, other choice set sizes as well as varying the configuration, and degree of similarity of the options within a choice set should be fruitful for understanding animal preferences in multialternative choice scenarios. For example, future experiments could test the prevalence of the simultaneous choice mechanism observed here in natural situations such as leks or in situations involving more than 3 alternatives (Hutchinson 2005). In general, however, our findings do pose a challenge that needs to be addressed simultaneously by mechanistic and functional modeling, as many natural problems probably involve multialternative tests of preferences, and looking at biological phenomena with the exclusion of either adaptive consequences or mechanistic implementation would always be unsatisfactory.
|
ACKNOWLEDGEMENTS |
|---|
We are grateful to W. J. Alonso, I. Cuthill, M. Elgar, E. Freidin, A. Grafen, A. Houston, J. Hutchinson, J. Krebs, D. Luce, M. Mangel, L. Pompilio, M. Root-Bernstein, M. Rossi, M. Shapiro, S. Siller, J. Stamps, and an anonymous reviewer for fruitful discussions and comments on an earlier version of the manuscript. Financial support was received from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior and Biotechnology and Biological Sciences Research Council Grant (43/S13483) to A.K.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODOLOGYDATA ANALYSISRESULTSDISCUSSIONREFERENCES |
|---|
Allen JA. Frequency-dependent selection by predators. Philos Trans R Soc Lond B Biol Sci (1988) 319:485–503.
Bateson M. Currencies for decision making: the foraging starling as a model animal [DPhil thesis] (1993) Oxford: University of Oxford.
Bateson M. Context-dependent foraging choices in risk-sensitive starlings. Anim Behav (2002) 64:251–260.[CrossRef][Web of Science]
Bateson M, Healy SD. Comparative evaluation and its implications for mate choice. Trends Ecol Evol (2005) 20:659–664.[CrossRef][Medline]
Bateson M, Healy SD, Hurly A. Irrational choices in hummingbird foraging behaviour. Anim Behav (2002) 63:587–596.[CrossRef][Web of Science]
Bateson M, Healy SD, Hurly A. Context-dependent foraging decisions in rufous hummingbirds. Proc R Soc Lond B Biol Sci (2003) 270:1271–1276.[Medline]
Bateson M, Kacelnik A. Preferences for fixed and variable food sources: variability in amount and delay. J Exp Anal Behav (1995) 63:313–329.[CrossRef][Web of Science][Medline]
Bateson M, Kacelnik A. Rate currencies and the foraging starling: the fallacy of the averages revisited. Behav Ecol (1996) 7:341–352.
Bateson M, Kacelnik A. Risk-sensitive foraging: decision making in variable environments. In: Cognitive ecology—Dukas R, ed. (1998) Chicago (IL): Chicago University Press. 297–341.
Brito e Abreu F. Risk sensitivity and decision making in birds [DPhil thesis] (1998) Oxford: University of Oxford.
Caraco T, Blanckenhorn WU, Gregory GM, Newman JA, Recer GM, Zwicker SM. Risk-sensitivity: ambient temperature affects foraging choice. Anim Behav (1990) 39:338–345.[CrossRef][Web of Science]
Cartar RV, Dill LM. Why are bumble bees risk sensitive foragers? Behav Ecol Sociobiol (1990) 26:121–127.[Web of Science]
Catania CA, Reynolds GS. A quantitative analysis of the responding maintained by interval schedules of reinforcement. J Exp Anal Behav (1968) 11:327–385.[CrossRef][Web of Science][Medline]
Charnov EL. Optimal foraging: marginal value theorem. Theor Popul Biol (1976) 9:129–136.[CrossRef][Web of Science][Medline]
Clarke FR. Constant-ratio rule for confusion matrices in speech communication. J Acoust Soc Am (1957) 29:715–720.[CrossRef][Web of Science]
Couvillon PA, Bitterman ME. Discrimination of color-odor compounds by honeybees: tests of a continuity model. Anim Learn Behav (1987) 15:218–227.[Web of Science]
Dow SM, Lea SEG. Sampling of scheduled parameters by pigeons: tests of optimization theory. Anim Behav (1987) 35:102–114.[CrossRef][Web of Science]
Gibbon J, Church RM, Fairhurst S, Kacelnik A. Scalar expectancy theory and choice between delayed rewards. Psychol Rev (1988) 95:102–114.[CrossRef][Web of Science][Medline]
Huber J, Payne JW, Puto C. Adding asymmetrically dominated alternatives: violations of regularity and the similarity hypothesis. J Consum Res (1982) 9:90–98.[CrossRef][Web of Science]
Hutchinson JMC. Is more choice always desirable? Evidence and arguments from leks, food selection, and environmental enrichment. Biol Rev (2005) 80:73–92.[Medline]
Kacelnik A, Bateson M. Risky theories—the effects of variance on foraging decisions. Am Zool (1996) 36:402–434.[Web of Science]
Kacelnik A, Brito e Abreu F. Risky choice and Weber's Law. J Theor Biol (1998) 194:289–298.[CrossRef][Web of Science][Medline]
Krebs JR, Kacelnik A, Taylor A. Tests of optimal sampling by foraging great tits. Nature (1978) 275:27–31.[CrossRef][Web of Science]
Leak TM, Gibbon J. Simultaneous timing of multiple intervals—implications of the scalar property. J Exp Psychol Anim Behav Process (1995) 21:3–19.[CrossRef][Web of Science][Medline]
Luce RD. Individual choice behaviour: a theoretical analysis (1959) New York: Wiley.
Mazur JE. Tests of an equivalence rule for fixed and variable reinforcer delays. J Exp Psychol Anim Behav Process (1984) 10:426–436.[CrossRef][Web of Science]
McNamara JM, Houston AI. Risk-sensitive foraging: a review of the theory. Bull Math Biol (1992) 54:355–378.[Web of Science]
Montague PR, Dayan P, Person C, Sejnowski TJ. Bee foraging in uncertain environments using predictive hebbian learning. Nature (1995) 377:725–728.[CrossRef][Medline]
Namboodiri NK. Experimental designs in which each subject is used repeatedly. Psychol Bull (1972) 77:54–64.[CrossRef][Web of Science]
Payne JW, Bettman JR, Johnson EJ. Behavioral decision research—a constructive processing perspective. Annu Rev Psychol (1992) 43:87–131.[CrossRef][Web of Science]
Real LA, Ott J, Silverfine E. On the tradeoff between mean and variance in foraging: an experimental analysis with bumblebees. Ecology (1982) 63:1617–1623.[CrossRef][Web of Science]
Reboreda JC, Kacelnik A. Risk sensitivity in starlings: variability in food amount and food delay. Behav Ecol (1991) 2:301–308.
Schuck-Paim C, Kacelnik A. Rationality in risk-sensitive foraging choices by starlings. Anim Behav (2002) 64:869–879.[CrossRef][Web of Science]
Schuck-Paim C, Pompilio L, Kacelnik A. State-dependent decisions cause apparent violations of rationality in animal choice. PLoS Biol (2004) 2:2305–2315.[Web of Science]
Shafir S, Waite TA, Smith BH. Context-dependent violations of rational choice in honeybees (Apis melifera) and gray jays (Perisoreus canadensis). Behav Ecol Sociobiol (2002) 51:180–187.[CrossRef][Web of Science]
Shapiro MS. Quantitative analysis of risk sensitivity in honeybees (Apis mellifera) with variability in concentration and amount of reward. J Exp Psychol Anim Behav Process (2000) 26:196–205.[CrossRef][Web of Science][Medline]
Shapiro MS, Couvillon PA, Bitterman ME. Quantitative tests of an associative theory of risk-sensitivity in honeybees. J Exp Biol (2001) 204:565–573.[Abstract]
Simonson I. Choice based on reasons: the case of attraction and compromise effects. J Consum Res (1989) 16:158–174.[CrossRef][Web of Science]
Simonson I, Tversky A. Choice in context: trade-off contrast and extremeness aversion. J Mark Res (1992) 29:281–295.[CrossRef][Web of Science]
Slaughter JE, Sinar EF, Highhouse S. Decoy effects and attribute-level inferences. J Appl Psychol (1999) 84:823–828.[CrossRef][Web of Science]
Smithson A, MacNair MR. Frequency-dependent selection by pollinators: mechanisms and consequences with regard to behaviour of bumblebees Bombus terrestris (L.) (Hymenoptera, Apidae). J Evol Biol (1996) 9:571–588.[CrossRef][Web of Science]
Stephens DW. The logic of risk-sensitive foraging preferences. Anim Behav (1981) 29:628–629.[CrossRef][Web of Science]
Tversky A. Elimination by aspects: a theory of choice. Psychol Rev (1972) 79:281–299.[Medline]
Tversky A, Russo JE. Substitutability and similarity in binary choices. J Math Psychol (1969) 6:1–12.[Medline]
Tversky A, Simonson I. Context-dependent preferences. Manag Sci (1993) 39:1179–1189.[CrossRef]
Wedell DH. Distinguishing among models of contextually induced preference reversals. J Exp Psychol Learn Mem Cogn (1991) 17:767–778.[CrossRef][Web of Science]
Wedell DH, Pettibone JC. Using judgments to understand decoy effects in choice. Organ Behav Hum Decis Process (1996) 67:326–344.[CrossRef][Web of Science]
Wunderle JM, O'Brien TG. Risk-aversion in hand reared bananaquits. Behav Ecol Sociobiol (1985) 17:371–380.[CrossRef][Web of Science]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||