Behavioral Ecology Vol. 12 No. 3: 330-339
© 2001 International Society for Behavioral Ecology
The adaptive value of preference for immediacy: when shortsighted rules have farsighted consequences
Ecology, Evolution and Behavior, University of Minnesota, St. Paul, MN 55108, USA
Address correspondence to D.W. Stephens. E-mail: dws{at}forager.cbs.umn.edu .
Received 20 December 1999; revised 23 August 2000; accepted 21 September 2000.
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Results from the operant laboratory suggest that short-term benefits guide animal feeding decisions. These results appear to contradict evolutionarily-motivated models of foraging that emphasize long-term benefits. Because of this contradiction, some behavioral ecologists argue that natural selection must favor short-term benefits in some unknown way. This study addresses the contradiction by testing the feeding preferences of captive blue jays in two economically equivalent situations. The first situation follows the operant literature's "self-control" paradigm; jays make a binary choice between small-immediate and large-delayed options. The second situation is modeled on patch-use problems; the jays make a leave-stay decision in which "leaving" leads to small amount in a short time, and "staying" leads to larger amount in a longer time. In the self-control situations, the observed outcome agrees with short-term rate maximizing, as other investigators have reported. In the patch situation, the results agree more closely with long-term rate maximizing. The text explains how a rule of preference based on short-term rate comparisons can account for both situations. It is argued that natural selection may have favored short-term rules because they have long-term consequences in many natural foraging situations.
Key words: blue jay, foraging, prey, self-control.
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
When a foraging blue jay detects a moth, it may attack or ignore it. If the jay successfully attacks, it obtains the moth after a delay involving pursuit and handling. If instead, it ignores the moth, it obtains its next meal at some later time. The jay's actions influence the timing and quantity of food. We can visualize this as a sequence of amounts arranged along a time-line (Figure 1), where Gi represents the size of the ith parcel of food gained, and ti represents the interval between the i-1st and ith bit of food. The environment constrains the forager's options—the forager can't simply choose small ti's and large Gi's. Foraging behavior interacts with the environment to determine a temporally structured stream of gains.
|
Since this is a ubiquitous problem, it seems reasonable to suppose that natural selection will have equipped animals with mechanisms that choose among time/amount sequences as shown in Figure 1 in some "sensible" way. This suggests two problems. First, how can we combine time and amount into an appropriate currency that correctly reflects the fitness consequences of different time/amount sequences? Second, what decision rules can foragers implement to select the currency-maximizing time/amount sequence from an environmentally determined set of possibilities? This article explores, via experiment and theory, the relationship between decision rules and currencies.
The next section reviews two contradictory points that form the background for our problem. First, evolutionary considerations suggest that foragers should choose among time/amount sequences in a farsighted way. Second, evidence from the operant laboratory contradicts this: subjects make short-sighted decisions; overemphasizing delay to the next food item, and under emphasizing the longer-term features of time/amount sequences. We review these issues in turn.
The logic of farsightedness
Foraging animals should evaluate the consequences of their actions over a
long time horizon, because there are likely to be many prey attack decisions
between each bout of reproductive activity (when the animal invests its
resources in reproductive success). The traditional
(Stephens and Krebs, 1986
)
long-term rate-maximizing currency reflects this logic:
 | (1) |
;
Baum and Rachlin, 1969) is:
 | (2) |
: 16, for a numerical example).
Rate models and discounting
The long- and short-term models [(1) and (2)] are both extremes, one
considers an infinite sequence of gains and the other considers only one. To
construct intermediate-term models, we can introduce a discount rate that
measures the proportional loss of value per s. If the discount rate is zero,
then we have the long-term model because gains have the same value regardless
of whether the forager obtains them early or late in a sequence. On the other
hand, as the discount rate becomes large, we approach the short-term
model.
We expect non-zero discounting for two reasons. First, there may be opportunity costs of delay (e.g., a forager loses the opportunity to use a delayed resource just as a human investor loses the opportunity to earn interest on a delayed payment). Second, delay implies a collection risk. That is, delay reduces the probability that the forager will collect a given benefit, because some intervening event—for example, an interruption by a predator or conspecific—might occur before the forager realizes the benefit.
While one certainly expects non-zero discount rates, it doesn't follow that the short-term model is more plausible than the long-term because discount rates are likely to be small. We expect small discount rates because we expect that the per-s rate of interruptions will normally be small, that is, fractions of an interruption event per s. Similarly, we expect that the amount of opportunity lost per s will also be small.
We will focus on the two rate models in this article, because they conveniently caricature the issues surrounding the time-scale of choice. Moreover, we argue that the long-term model makes the most evolutionary sense because: it gives a reasonable approximation of rate even when the time horizon is not infinite, it is farsighted enough to skip over bad options as explained above, and realistic discount rates are likely to be small.
Evidence of shortsightedness
Notwithstanding our argument that foragers should make farsighted
decisions, an impressive body of experimental evidence shows that animals
often make myopic decisions. Most of this evidence comes from
"self-control" experiments in the operant psychology lab.
Figure 2 diagrams a single
trial from a typical self-control experiment. The animal waits for time 
,
and the apparatus presents a binary choice. The subject can choose either the
small-immediate (wait t1 s to obtain amount G1), or the
large-delayed (wait t2 seconds to obtain amount G2;
t2 > t1 and G2 > G1) option.
Typically, the subject has had prior experience that allows it to associate
(for example) a red key with the small-immediate choice and a blue key with
the large-delayed choice. In some experiments, there is a postfeeding delay
(
in Figure 2) associated
with one or both of the options. The investigator varies the amounts and
delays, and records the proportion of choices made to each alternative.
Psychologists say that a subject exhibits self-control if it waits for the
larger delayed amount.
|
It is useful to consider the long-term rate models predictions for the
self-control situation (Figure
2). The rate associated with choosing alternative 1 for many
trials is:
 |
 |
 |
The delay to food
Investigators universally agree that the time between when a choice is made
and food is delivered (t1 and t2 in
Figure 2) is a key determinant
of animal preferences. There are several pioneering studies in this area,
including those of Rachlin and Green
(1972) and Ainslie
(1974). In these studies and
others, direct manipulation of delay affects preference in a powerful and
consistent way. For example, McDiarmid and Rilling
(1965) showed that pigeons
preferred a schedule with a 6 s delay to initial food delivery to an option
with a 24 s delay to food, even though the 24 s option delivered twice as much
food over a 2 min period.
Post-feeding delay
Several studies have shown that post-feeding delays (p in
Figure 2) have virtually no
effect on preference (e.g., Green et al.,
1981; Logue et al.,
1985; Mazur and Logue,
1978; Snyderman,
1987). This challenges our view that animal choices should be
farsighted, because the delay to food (t1) and the post-feeding
delay (p) should be equivalent from a "long-term" perspective.
Inter-trial interval
Most investigators feel that the inter-trial interval (ITI), like the
post-feeding delay, has little effect on preference. Mazur
(1989) manipulated the ITI and
found virtually no effect on preference. Bateson and Kacelnik
(1996) performed a preference
experiment, and compared their data to several ratebased models. They found
that a model excluding the ITI, (i.e., maximize
Gi/ti) gave the best fit.
The strength of animal preferences for immediacy also disagrees with
farsighted models. For example, hyperbolic models of discounting, derived from
the self-control data (Ainslie,
1975; Mazur, 1984,
1986,
1987;
Mazur et al., 1985), suggest
that the first s of delay can cut value in half. While several discounting
processes probably reduce the value of delayed benefits, it is hard to imagine
plausible processes that could justify such severe discounting.
Summary: the contradiction
Evolutionary models disagree with self-control results. They disagree
qualitatively, for example, over the role of the intertrial interval, which is
important in the long-term calculations of evolutionary models, but relatively
unimportant in self-control experiments. They also disagree quantitatively
over the strength of preference for immediacy. While evolutionary models can
accommodate small discounting effects, self-control experiments show that
brief delays can have large effects.
How should we interpret this disagreement?
When a model disagrees with data, we must accept the data and reject the
model. Following this principle, many behavioral ecologists (e.g.,
Bateson and Kacelnik, 1996;
Stephens, 1996) have taken
self-control results to mean that our traditional "long-term"
currencies must be wrong, concluding that natural selection must have
emphasized short-term consequences in some way that we do not understand. The
problem here is that it is difficult to construct a reasonable economic model
that predicts the strong preferences for immediacy that self-control studies
show. For example, a model offered by Green and Myerson
(1996) assumes that the rate
of interruptions increases smoothly and systematically after a forager
encounters a stimulus predicting a delayed food reward. While this correctly
predicts some features of the self-control data, the only justification for
this assumption is that it fits the data. There are no interruptions,
decreasing or otherwise, in typical self-control studies.
Is there some other way to resolve the conflict? The simplest alternative is that self-control data are not an accurate guide to the economic forces that have shaped choice. Specifically, although the overall pattern of preference for immediacy is impressive and highly repeatable within the self-control paradigm, does it necessarily mean that animals always make myopic choices? Could the decision-making mechanism that seems so frustratingly myopic in self-control situations lead to more understandable farsighted consequences in other situations?
Self-control and patch-use
We were attracted to this second possibility by an empirical conflict
between the self-control and patch-exploitation literatures. To understand
this conflict, the reader must first understand the parallel between the
patch-exploitation and self-control. The elementary theory of patch
exploitation (the marginal-value theorem, see
Charnov, 1976;
Stephens and Krebs, 1986)
considers the effect of patch residence time on the long-term rate of food
intake. Figure 3A shows a
graphical analysis of this problem. This well-known plot shows how travel time
affects the rate-maximizing patch residence time. This model predicts that
foragers should spend more time in patches extracting more resources when
travel times are longer.
|
When a forager exploits a food patch it makes a decision that is analogous
to a self-control experiment, because it must decide whether to spend a short
time in the patch obtaining a small amount, or spend a longer time obtaining
more. To emphasize this similarity, we can plot a self-control experiment's
small-immediate and large-delayed options in a time/gain space that parallels
the classic marginal-value theorem plot,
(Figure 3B;
Stephens et al., 1986).
Moreover, we see that travel time, which is analogous to the ITI, should have
an effect on preference: as the travel time (ITI) increases animals should
shift their preference from the small-immediate to the large-delayed option.
Theoretically, this restates our claim that farsighted foragers should be
sensitive to the ITI. Empirically, however, it raises a new question: why do
patch experiments consistently show animals spending longer to extract more
when travel times are long (Stephens and
Krebs, 1986), when self-control experiments show little effect of
the analogous ITI?
Experimental rationale
This empirical disagreement stimulated our study. We realized that while
patch exploitation and self-control problems are similar economically, they
differ in many ways. For example, in a self-control experiment, the animal
makes a binary choice—left or right, red or green—but in patch
exploitation, the animal chooses whether to continue or to leave and pursue
the option of a future patch. Figure
4 shows economically identical patch and self-control problems
that we used in our experiment. In the self-control problem the forager simply
chooses either option 1 or 2. In the patch problem the forager waits (travels)
for time and then encounters a single stimulus (a patch), it then waits
t1 s to obtain G1. Now, it faces a choice, it can
"leave" the patch, starting an new cycle of travel and patch
encounter, or it can continue for further time t2 - t1
to obtain an additional G2 - G1. If the forager takes
the "leave" option it gains G1 in total time +
t1. If it takes the "stay" option, it obtains
G2 in + t2. These, of course, are the same time
and amount consequences (or equivalently, the same two-point gain function,
Figure 3B) as in the
self-control problem.
|
By constructing economically equivalent patch and self-control situations, our experiment considers the economic significance of self-control results. If, as some have supposed, short-term consequences shaped the mechanisms of choice in a general way, then we should observe the same patterns of choice in patch and self-control situations. If, instead, we find that patch and self-control results disagree, this will suggest that self-control results are not a reliable guide to the economic forces that have shaped animal preferences.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Overview
Our experiment tests foragers in "equivalent" patch and self-control contexts as shown in Fig. 4. Since empirical evidence shows an effect of ITI in patch studies, but not in self-control studies, we tested each context at three different ITI's (30 s, 60 s, and 90 s). Table 1 shows our experimental design. Within each combination of context (patch or self-control) and ITI we assessed preferences via factorial tests of delay-to-large (DTL = t2) and delay-to-small (DTS = t1) as shown in Table 1. The notation P(L) in Table 1 indicates that we used proportional choice of large (P(Large)) as our measure of preference. We used four 20-mg food pellets for the large amount (G2 = 4) and two food pellets for the small amount (G1 = 2).
|
Procedural details
The subjects were six adult blue jays (Cyanocitta cristata) of
unknown sex and mixed experimental histories.
Figure 5 shows a top view of
the apparatus. The apparatus has two operational stimulus lights and three
perches equipped with microswitches. Birds exhibited their preferences by
selecting which perch to hop on at a given time. We ran the experiment as a
modified closed economy; the jays spent 23 h per day in the apparatus and
obtained all of their food there. The experimentally programmed contingencies
were in force from 0700 to 1600 CST daily, with a 1-h break from 1100 to 1200.
During the break, we removed each bird from the apparatus, weighed it, and
cleaned the apparatus. A white noise generator provided masking noise. A clock
turned the house lights on at 0600 and off at 1800 daily.
|
Table 1 shows the temporal parameters for each treatment. The experiment followed a within-subjects design with each jay experiencing all combinations of the treatment variables. A process of nested randomizations determined the order of treatments. Each bird completed a full set of treatments in one decision context (patch or self-control) before changing to the other; we assigned three birds to "self-control first" and three to "patch first." Before beginning a new context, the birds experienced 7 days of baseline treatments with the temporal parameters (delays and ITI) set to the averages of the experimental parameters; this was an attempt to minimize the effects of learning a new choice paradigm in the transition from self-control to patch or vice-versa. We randomized the order of ITI treatments within each decision context. Similarly, we randomized the order of each delay-to-large treatment within each decision context/ITI cell, and we randomized the order of delay-to-small treatments within each delay-to-large block.
We assigned a pair of stimulus colors to each bird for each context. We assigned blue-yellow to patch, and red-green to self-control for three of the birds, and vice-versa for the remaining three. The meaning of a color (e.g., red associated with small-immediate), was assigned randomly but was consistent throughout a context treatment.
Training
We trained subjects using shaping (sometimes called the method of
successive approximations). We considered an individual trained when it
reliably performed a task we called light-following. In the light-following
task, the bird sits on the rear perch, until the computer switches on one of
the two front stimulus lights; then the bird hops forward and collects a few
food pellets when it lands on the front perch adjacent to the illuminated
light.
Within trial details
The following paragraphs explain how we implemented the self-control and
patch contexts.
Self-control. During the ITI, no stimuli were present and birds typically waited on the rear perch. After the ITI expired, the computer switched on a pair of stimulus lights near the front perches if the bird was waiting on the rear perch. The bird then chose a side by hopping forward. When the bird made its choice, the computer switched off the unchosen light, and "washed out" the color of the chosen stimulus light (by turning on two clear bulbs in the stimulus projector; this signaled the "waiting" period). The computer dispensed the pellets after the programmed delay elapsed if the bird was on the food perch. A hop on the rear perch started the next ITI.
Patch. The ITI phase was identical to self-control. After the ITI expired, the computer presented the stimulus color associated with the small amount on a randomly chosen side of the apparatus, (again, the bird must have been on the rear perch for this presentation to occur). When the bird hopped forward, the "small color" was washed out, and the programmed delay began. When the programmed small delay expired, the computer dispensed two pellets and the color associated with the large was displayed for 1 s and then washed out; the programmed delay to large (t2 - t1) began when the first two pellets were dispensed. If the bird was on the front perch when the programmed delay expired, then the remaining two pellets were dispensed. At any time after the initial hop forward, a hop on the rear perch canceled the trial and started a new ITI; that is, the bird was free to "leave the patch" at any time.
Further details. We organized sequences of trials into blocks of 32. Each block began with eight forced-choice trials (in which subjects were forced to take either the large or small option) and 24 free-choice trials from which we collected preference data. The purpose of the forced-choice trials was to insure that subjects had some experience with each option. The most basic element of our design was a simple preference test for a fixed context, ITI, delay-to-large and delay-to-small. In each preference test, we recorded the number of times the subject obtained the larger four pellet amount. Each preference test lasted for 3 days. We used relative frequency of choice in the last third of the free trials as our estimate of preference, (typically, there were between 100 and 120 trials in this "last third"). We call this measurement "P(Large)."
We would have preferred to use a stability criterion to terminate each preference test, but the large number of tests required (24 for each individual) made this impractical. The possible problem with our fixed duration tests is that both preference (which we want to measure) and rates of preference acquisition (which we don't want to measure) can affect the outcome. However, the general agreement between our results and well documented self-control procedure (see results and discussion) leads us to think that we have reasonable measures of preference.
Latencies and the control of timing. We did not have absolute
control over the timing of events in this experiment, because jays seldom
respond immediately when the apparatus offers a choice. This means that
realized intervals are always longer than programmed intervals. This could be
a special problem here, because we tried to create equivalent self-control and
patch situations. If, for example, latencies to respond were consistently
longer in the patch situation, then this would undermine the logic of our
experiment. To consider this possibility, we calculated the difference between
programmed trial durations and realized trial durations for every free choice
trial. We summarized this data by calculating mean differences for each
subject in each condition, and then performed a repeated-measures analysis of
variance (like that shown in Table
2) using this mean differences as the dependent measure. This
ANOVA showed no significant (5% level) effects (e.g., effect of context,
F1,5 = 0.35, P 
0.58).
|
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Self-control experiments show strong preferences for immediacy that have lead behavioral ecologists to question their ideas about the fitness value of food gains. Our experiment explores the generality of these results by asking whether blue jays choose short-term consequences in equivalent patch and self-control situations.
Overview
Figure 6 shows an overview
of our results. The figure shows different patterns of choice in the
self-control and patch contexts. To describe these differences, we consider
the 50 and 5 s levels of delay-to-small separately. When the delay-to-small
was 50 s (the right half of Figure
6), the ITI had no effect on preference, but the jays were more
likely to choose the large outcome in the patch context (mean P(Large) for
patch: 76%, mean for self-control 48%). When the delay-to-small was 5 s (the
left half of Figure 6) the
results were more complex. As expected, preference for large was greatly
reduced (mean P(Large) for patch: 16%, mean for self-control 21%), but the
most striking feature of these data is an interaction between context and ITI.
In patch, preference for large increased with ITI, while in self-control
preference for large decreased with ITI.
|
Table 2 presents a partial
analysis of variance table that supports this interpretation. The dependent
measures in this ANOVA are arcsin transformed versions of the P(Large) data
(Figure 6). The analysis
follows Myers and Well's
(1995) recommendations for
within-subjects (repeated measures) ANOVA.
Table 2 only shows sources of
variation with significant effects; all other terms were not significant. The
pattern of significant effects shown in
Table 2 supports the following
interpretation. There is simple additive effect of delay to large (DTL);
larger delays shift preference away from the large option, as one expects.
Delay to small (DTS) interacts in a complex way with inter-trial interval
(ITI) and context as Figure 6
for delays-to-small of 5 s there is an ITI/Context interaction with preference
for large increasing with ITI in the patch context, but decreasing with ITI in
the self-control context. For delays-to-small of 50 s, however, there is no
effect of ITI but a strong additive effect of context.
ITI effects
The interaction of ITI and context (when delay-to-small was 5 s) is a
striking result, but the trends shown in the grouped data
(Figure 6) are subtle. So, a
critical reader may wonder whether this result holds within subjects. Although
preference does vary from one individual to the next,
Figure 7 shows that the effect
of ITI is remarkably consistent. In self-control the birds had stronger
preferences for large when the ITI was small (with one exception; Bird 70,
Delay to large = 60 s). In patch the birds had a stronger preference for large
when the ITI was large (without exception).
|
Broadly speaking, long-term theory predicts increasing preference for large with increasing ITI. In considering the patch data, one is bound to wonder why we see the predicted pattern when the delay-to-small is 5 s, but not when the delay-to-small is 50 s. Long-term theory does not predict that preference for large will always increase with ITI, because proportional choice cannot exceed 100%. This means that there must be broad conditions where the ITI has no effect. The patch data agree with this idea, the ITI effect is most striking when overall preference for large is low, but difficult to detect when preference for large is high. We know of no rationale for the decreasing preference for large that we observed in some self-control conditions. It is difficult to assess the novelty of this result, because direct manipulation of ITI is relatively rare in self-control studies.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
We tested situations where a leave-stay choice had the same consequences as a binary, self-control-style choice. The data show different outcomes in these different choice contexts. Specifically, blue jays switched to large-delayed outcomes more readily in the patch situation. In addition, preference for large increased with ITI as predicted by long-term theory in the patch situation, but not in the self-control situation.
Behavioral ecologists have accepted self-control data as evidence against the long-term rate currency of elementary foraging theory. Our results challenge this view, because they show that animals do not exhibit the same preferences in an economically equivalent situation. Which situation provides the most accurate picture of the forces that have shaped animal feeding preferences? In what follows, we develop a hyothesis that resolves the conflict between self-control experiments and long-term models. Our development proceeds in two steps. First, we compare our patch and self-control treatments to the long and short-term models, outlined in the introduction. Second, we show how the same decision rule can, in theory, lead to long-term rate maximizing in the patch situation, and the characteristically myopic choice of self-control.
The rate models
To begin, we compare our results to the farsighted long-term rate model
(Equation 1) and the myopic short-term rate model (Equation 2), because these
models represent different ways to value temporal arrangements of
benefits.
Long-term model
The currency of long-term rate maximization predicts preference for the
large (four pellet) alternative when:
 |
is the ITI and DTL and DTS represent the delays to large and
small, respectively. This calculation assumes that four pellets are twice as
valuable as two pellets. In reality, the relationship between amount and value
will be non-linear. We assume a linear relationship here, so that we may focus
on the temporal elements of the problem.
Now, consider the difference
 | (3) |
LTR is negative, the small option yields a higher long term rate of
gain; when it is positive, the large option gives the higher rate. A naive
version of the long-term rate model predicts 0% choice of large when LTR
is negative, and 100% choice of large when LTR is positive. This step
function prediction is unrealistic, however, because many sources of
variability affect animal behavior. Instead, one expects a sigmoid
dose-response curve. Figure 8
plots LTR versus P(Large) for the patch and self-control contexts. The
figure shows a well defined sigmoid relationship in the patch context, and a
more amorphous increasing relationship in the self-control context.
Figure 8 also shows the
least-squares fits for probits-style doseresponse curves (which assume that
the response threshold follows a normal distribution, see
Finney, 1962;
Stephens, 1985, for a
discussion of the probits technique). The patch data gives a better fit to
this standard dose-response model in two ways. First, the patch data give a
higher R2 value (0.61 for patch and 0.19 for self-control). Second,
the calculated indifference threshold is much closer to the predicted value of
zero in the patch data (0.0052 for patch, 0.021 for self-control). We
conclude, therefore, that the outcome in the patch context agrees, at least
crudely, with the long-term rate model, while the self-control treatment does
not.
|
Short-term model
Next, we consider the short-term rate model suggested by Bateson and
Kacelnik (1996, and supported
by much of the self-control literature). This model predicts preference for
large when:
 |
 |
STR
calculations for our experiment. The table shows two clusters of STR.
When the delay to small is 5 s, STR is about -1/3 predicting a
relatively strong preference for the small, two pellet, option. However, when
the delay to small is 50 s STR values are near zero, predicting
indifference or a weak preference for large. These predictions agree with the
results of our self-control treatments, where we observed preference for small
when DTS = 5, and approximate indifference when DTS = 50. In contrast, jays in
the patch treatment show a clear preference for the large alternative in the
DTS = 50 condition.
|
STR
Our self-control treatment agrees with the results of other binary choice
studies (e.g., Bateson and Kacelnik,
1996; Mazur, 1987;
Mazur et al., 1985) that
support short-term rate models. In a different, economically equivalent,
choice situation modeled on patch exploitation, we find that the outcome
agrees more closely with the long-term rate currency.
The "same rule/different outcomes" hypothesis
While the outcomes differ in the patch and self-control contexts, it does
not necessarily follow that the jays use different choice rules in the
different contexts. The jays might use a rule that compares
short-term rates in both cases; such that these short-term rate comparisons
achieve long-term rate maximization in the patch context, but not in
self-control.
To see why our economic result does not imply a farsighted rule or even a
different rule, consider the general form of the short-term rule:
 | (4) |
Consider, however, how one might apply this rule to the patch context (cf.
Figure 4): after obtaining the
small amount G1 the subject must choose between "starting
over" which leads to G1 after a delay of +
t1, and "continuing" which leads to G2 -
G1 after a delay of t2 - t1. The short-term
rule (4), therefore, predicts:
 |
 | (5) |
STR is the difference in short-term rates. Applying a short-term
rule in the patch context predicts sensitivity to the ITI-like term
(because it is part of a key delay). Some algebra reveals a more startling
result:
 | (6) |
STR), and the right-hand side is the difference in long-term rates
(LTR). In our patch context, then, the short-term rule always
agrees with, and even amplifies (because (t2 -
t1)/( + t2) < 1), the difference in long-term
rates; even though the same rule leads to notoriously short-sighted behavior
in the self-control context. The simplest account of our results, therefore,
is that jays use the same shortterm rule in both situations, and this
short-term rule approximates long-term maximizing in the patch situations, but
not in the self-control situation.
Adaptive short-term rules
This suggests the following hypothesis. Natural selection has favored a
mechanism based on short-term rate comparisons because these rules approximate
long-term rate maximization in many natural decision contexts. Prey choice
situations involving binary, mutually exclusive choice (as in self-control
studies) are probably rare in nature, so animals may not be equipped to behave
appropriately in these situations. While self-control studies give an accurate
view of the rules of choice; we must consider the natural contexts of choice
behavior, if we are to understand the adaptive value of these rules. The idea
that "local rules" can achieve global consequences is not new.
Among others Staddon (1983)
emphasized a link between local rules and global consequences. Bateson and
Kacelnik (1996) used similar
reasoning to explain why there is an "ITI/travel time" effect in
patch studies and not in self-control experiments.
This hypothesis raises several questions. First, the claim that foragers use the same rule in patch and self-control contexts can be tested directly. For example, an investigator might arrange situations where a decision "to leave" implies the same delay and amount as the "large" alternative in a parallel self-control treatment, while the decision to stay has the same time/amount consequences as the "small" alternative. Second, our patch condition represents a narrow slice of "natural" foraging problems. There are many other situations to explore; for example, patches with more than two food items; or diet choice problems where a forager must choose whether to attack or ignore sequentially presented food items.
Why short-term rules?
We argue that long-term rate rules, and short-term rate rules accomplish
the same thing in patch use (and possibly other) situations. That is, they are
equivalent adaptive peaks. Why, then, should animals use short-term rules
instead of longterm? There are three possibilities. (1) It may be completely
haphazard. The two rules are equivalent and one arose instead of the other via
simple randomness. (2) Alternatively, it may reflect a feature of the neural
and sensory mechanisms of choice. The neural machinery may be better suited to
short-term rules, because common neural motifs such as opponent-processing may
adapt more readily to a comparison of local rates. (3) Finally, short-term
rules may have some advantage that we do not yet appreciate. There may, for
example, be a psychophysical advantage as Equation 6 suggests, because
short-term comparisons can make long-term differences more detectable. In
addition, short-term rules might be advantageous because they reflect the
small effects of discounting (collection risk, and lost opportunity risk).
These possibilities are not, of course, mutually exclusive.
|
CONCLUSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
There have been two lines of evidence against the long-term rate currency of foraging theory: the strong preferences for immediacy seen in self-control studies, and data demonstrating so-called risk-sensitive preferences. Studies of risk-sensitivity show that animals prefer variable delays (see Kacelnik and Bateson, 1996
,
for review) even though infinite time-horizon rate models predict indifference
to variability. We have argued that the self-control literature's evidence of
short-term rules is not necessarily evidence of a short-term currency, because
short-term rules can have long-term consequences in, at least some, natural
decision contexts. We are not, however, trying to rehabilitate the long-term
rate currency of classical foraging theory, because the risk-sensitivity data
argues against it, and because there are a priori reasons to reject its
infinite time-horizon. What alternative do we offer? We hypothesize that a
long, but not infinite, term rate currency incorporating plausible levels of
discounting will account for observed patterns of preference. A currency,
however, is not enough. We must also understand how this currency has shaped
the rules of choice in specific natural foraging situations. This suggests
research focused on how currency and context have combined to shape decision
rules. |
ACKNOWLEDGEMENTS |
|---|
We thank William Baum, Leonard Green, James Mazur and Howard Rachlin for helpful discussions and comments on a preliminary presentation of these results; and Jeff Stevens and Colleen McLinn for comments on the manuscript. In addition, we thank David Westneat for thoughtful editorial suggestions and encouragement. The National Science Foundation supported this research (IBN-95077680 and IBN-9896102), and we are grateful for this support.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Ainslie, G, 1974. Impulse control in pigeons. J Exp Anal Behav 21: 485-489.[ISI][Medline]
Ainslie, G, 1975. Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol Bull 82: 463-496.[ISI][Medline]
Bateson M, Kacelnik A, 1996. Rate currencies and the
foraging starling: the fallacy of the averages revisited. Behav
Ecol 7:
341-352.
Bateson M, Whitehead SC, 1996. The energetic cost of alternative rate currencies in the foraging starling. Ecology 77: 1303-1307.
Baum W, Rachlin H, 1969. Choice as time allocation. J Exp Anal Behav 12: 861-874.[Medline]
Benson K, Stephens DW, 1996. Interruptions, tradeoffs and temporal discounting. Am Zool 36: 506-517.
Charnov EL, 1976. Optimal foraging: the marginal value theorem. Theor Popul Biol 9: 129-136.[ISI][Medline]
Cleveland WS, 1993. Visualizing data. Summit, New Jersey: Hobart Press.
Finney DJ, 1962. Probit analysis. New York: Cambridge University Press.
Green L Jr, Fisher EB, Perlow S, Sherman L, 1981. Preference reversal and self-control: choice as a function of reward amount and delay. Behav Anal Lett 1: 244-256.
Green L, Myerson J, 1996. Exponential versus hyperbolic discounting of delayed outcomes: risk and waiting time. Am Zool 36: 496-505.
Kacelnik A, Bateson M, 1996. Risky theories—the effects of variance on foraging decisions. Am Zool 36: 402-434.
Logue AW, Smith ME, Rachlin H, 1985. Sensitivity of pigeons to prereinforcer and postreinforcer delay. Anim Learning Behav 13: 181-186.
Mazur JE, 1984. Tests of an equivalence rule for fixed and variable reinforcer delays. J Exp Psychol: Anim Behav Proc 10: 426-436.[ISI]
Mazur JE, 1986. Choice between single and multiple delayed reinforcers. J Exp Anal Behav 46: 67-77.[ISI][Medline]
Mazur JE, 1987. An adjusting procedure for studying delayed reinforcement. In: Quantitative analyses of behavior: the effect of delay and of intervening events on reinforcement value (Commons ML, Mazur JE, Nevin JA, Rachlin H, eds.). Hillsdale, New Jersey: Lawrence Erlbaum Associates; 55-73.
Mazur JE, 1989. Theories of probabilistic reinforcement. J Exp Anal Behav 51: 87-99.[ISI][Medline]
Mazur JE, Logue AW, 1978. Choice in a "self-control" paradigm: effects of a fading procedure. J Exp Anal Behav 30: 11-17.[Medline]
Mazur JE, Snyderman M, Coe D, 1985. Influences of delay and rate of reinforcement on discrete-trial choice. J Exp Psych: Anim Behav Proc 11: 565-575.[ISI][Medline]
McDiarmid CF, Rilling ME, 1965. Reinforcement delay and reinforcement rate as determinants of schedule preference. Psychonomic Sci 2: 195-196.
Myers JL, Well AD, 1995. Research design and statistical analysis. Hillsdale, New Jersey: Lawrence Erlbaum and Associates.
Rachlin H, Green L, 1972. Commitment, choice and self-control. J Exp Anal Behav 17: 15-22.[ISI][Medline]
Snyderman M, 1987. Prey selection and self-control. In: Quantitative analyses of behavior: the effect of delay and of intervening events on reinforcement value. (Commons ML, Mazur JE, Nevin JA, Rachlin H, eds.). Hillsdale, New Jersey: Lawrence Erlbaum Associates.
Staddon JER, 1983. Adaptive behavior and learning. New York: Cambridge University Press.
Stephens DW, 1985. How important are partial preferences? Anim Behav 33: 667-669.
Stephens DW, 1996. On the relationship between the feedback control of digestion and the temporal pattern of food intake. Biol Bull 191: 85-91.[ISI][Medline]
Stephens DW, Krebs JR, 1986. Foraging theory. Monographs in behavior and ecology. Princeton, New Jersey: Princeton University Press.
Stephens DW, Lynch JF, Sorensen AE, Gordon C, 1986. Preference and profitability: theory and experiment. Am Nat 27: 533-553.
Stephens DW, Nishimura K, Toyer KB, 1995. Error and discounting in the iterated prisoner's dilemma. J Theor Biol 76: 457-469.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. E. Henly, A. Ostdiek, E. Blackwell, S. Knutie, A. S. Dunlap, and D. W. Stephens The discounting-by-interruptions hypothesis: model and experiment Behav. Ecol., January 1, 2008; 19(1): 154 - 162. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Ohashi, J. D. Thomson, and D. D'Souza Trapline foraging by bumble bees: IV. Optimization of route geometry in the absence of competition Behav. Ecol., January 1, 2007; 18(1): 1 - 11. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||