The Clinical Utility of Utility Assessment
Harold Bursztajn, M.D., and Robert M. Hamm, Ph.D.
From the Program in Psychiatry and the Law, Massachusetts Mental Health
Center, Boston, Massachusetts, and the Center for Research on Judgment
and Policy, University of Colorado, Boulder, Colorado. Dr. Bursztajn’s
participation in this work was supported in part by NIMH Grant #TOTMH64603.
Dr. Hamm was a research fellow in the Division of Primary Care and Family
Medicine, Harvard Medical School.
An earlier version of this paper was presented to the First Annual Meeting
of the Society for Medical Decision Making, Cincinnati, Ohio, September
12, 1979. Please address requests for reprints to Dr. Bursztajn, Program
in Psychiatry and the Law, Massachusetts Mental Health Center, 74 Fenwood
Road, Boston, Massachusetts 02115, USA.
Abstract
The usefulness of utility assessment as a method for revealing individual
patients’ desires is limited by two methodological problems. Different
utility assessment methods can yield inconsistent results, both within
a single clinical context and in different contexts; and the methods
may not reflect universal rules that a patient may wish to abide by.
To make utility assessment useful to the clinician, future research needs
to address these problems in the clinical context. (Med Decis Making
2:161-165, 1982)
Article
A variety of current trends express the need for greater patient participation
in decision making. An increasingly influential response to this need
is to use utility assessment procedures to measure individual patients’
preferences for the outcomes of possible treatments. For example, McNeil,
Weichselbaum, and Pauker [1] used a utility assessment
procedure to measure individual patients’ preferences for duration of
life, indicating which of two treatments for lung cancer would be preferred
-radiation or surgery. This is an important demonstration of the use
of utility theory in medical decision making. However, before widespread
use of such a utility measure as a means for ascertaining patient values
would be justified, further questions about its reliability and validity
need to be answered. The sorts of questions that one must ask can be
clarified by examining the version of the lottery method of subjective
utility assessment [2] that is commonly used, as in
the case of the paper by McNeil et al. This method produces a &dquo;utility
curve,&dquo; showing the relation between the amount of life remaining
and the subjective value of that amount of life. We ask, first, whether
it is wise to rely on only one such method to measure utility?
Poulton [3] has shown that when asking for subjective
judgments of sensory magnitudes, the particular details of the methods
can have major effects on the subjective relation revealed. For example,
when judging the loudness of sounds, asking for judgments of stimuli
near some limit, such as the threshold of hearing, may give a different
overall relation than if such quiet stimuli are not considered, because
the subjective impression of those stimuli is different near the threshold.
Looking for analogous effects in the assessment of utility, we wonder
whether a lottery method that uses a 50% chance of dying in one week
as its zero point might not yield utility curves less risk averse than
one that anchors on a 50% chance of "dying immediately" [1].
We note that in a later work McNeil, Weichselbaum, and Pauker [4] have
sought to avoid this problem by offering 50-50 gambles between survival
for one’s full life expectancy and death in a few months. Another method
would be to partition the range of outcomes into subranges that the patient
finds it natural to think about, assess the utility function over each
range, and then combine the functions into one function for the whole
range, in the manner of Krzysztofowicz and Duckstein [5] and
Farquhar and Fishburn [6].
It has been shown that it makes a difference whether people indicate
preference by taking a choice between two alternatives (as do McNeil,
Weichselbaum, and Pauker’s patients) or by directly naming numbers to
represent utility. The systematic inconsistencies found between the two
methods [7, 8] give us reason to consider
very carefully what methods we want to use. Recent work showing how different
utility scales are produced by different lottery method procedures [9-12] supports
this recommendation.
A second major problem is that it is not easy to bring all reasonably
relevant considerations into discussions with patients. How may this
be done in such a way as to guarantee the validity of the utilities of
remaining life measured by a method such as a 50-50 lottery? First, we
must expect that fear and denial [13] would be manifest
in patients with severe illness of recent onset. These could make patients
either more or less risk averse than they might be under other conditions.
Such patients might not think of certain important considerations such
as the impact that their health care choices might have on the well-being
of others, including their families, loved ones, and the community as
a whole. They might give a place of specious prominence to other, less
important considerations. For example, one’s self-interest may come to
be far more narrowly conceived, excluding altruism and concern for others,
as well as being given a greater emphasis than it had prior to the procedure [14].
These effects can happen with either the lottery or the direct assessment
method. To counter these, a method for clinical use should have some
of the characteristics of a well-done psychiatric interview that, by
providing a supportive relationship, seeks to free the patient from overwhelming
fear and aloneness in the face of illness. The hope would be that the
patient might thereby have freer access to those values which can be
most easily expressed when one is secure enough to function at one’s
most mature level [15, 16].
There are also certain preferences some patients have which may be quite
difficult to express with these utility assessment methods. A patient
may think that the decision in his or her case should be made according
to universal considerations, such as those expressed in religion and
philosophy, rather than according to the chaos of one’s momentary impulses.
If one wants one’s behavior to be governed by universal rules, it may
not be immediately clear how to express this using the lottery method.
Suppose that a patient may have a notion that it is good to die quickly,
cleanly, and with the dignity that is often attributed to operative death.
It is not obvious to an untutored patient (or indeed to us) how to express
these values in terms of the lottery method, when its only parameters
are time of life left. Though it may be possible to express this universal
rule through the lottery method, can we expect the patient to be able
to do this well? Perhaps training in using the lottery method to express
principles would solve this problem, or again a special procedure modeled
on the psychiatric interview could be undertaken to elicit the patients’
principles and explore how these would be reflected in the utility assessment
method.
It might well be possible to use the lottery method to express the preferences
embodied in most universal rules. However, there are some universal rules
that could militate against the use of the 50-50 lottery method itself.
A rule against suicide, for example, might make it difficult for a patient
to consider taking a gamble with a 50% chance of immediate death. This
bias against a particular aspect of one of the alternatives would make
the patient appear more risk averse than another method might reveal.
Special procedures are needed to elicit the universal rules that patients
may wish to use, so as to avoid the automatic application of assessment
methods in a manner that undermines the results and defeats the purpose
of the utility assessment.
In order for clinicians to be able to apply utility theory by using the
results of work such as the study by McNeil et al. [1],
it is necessary that the methods be embedded in a repertoire which offers
a variety of methods and justifications for their selection. Why is one
utility assessment procedure more appropriate than another? Justifications
for selecting a particular method would depend on evidence that other
available methods will not yield substantially different results, or
if they do, there must be some basis for preferring either the method
or the pattern of results that the chosen procedure produces.
The research necessary to create such a repertoire would involve comparing
the results of the different methods in each of a variety of situations.
The results of this comparison, embodied in the rules of thumb for selecting
from the repertoire, could alert the user to possible pitfalls and systematic
reversals of preference engendered by the use of particular methods in
particular situations.
As a final caveat, a repertoire of values assessment methods would be
incomplete if, when using experimental comparisons to construct it, the
"method of no method," i.e., the clinician’s intuitive approach,
were not considered. The clinician’s approach offers a valuable comparison,
not just as a "control" to see whether these systematic utility
assessment methods are any better than "no method at all," but
because it would not have the systematic errors that can sometimes result
from using a particular elicitation procedure. Such a critical balance
between the systematic and the intuitive approaches to assessment of
patients’ values is needed. "Professional students of society ...
often make large errors of judgment because of an overly analytical and
doctrinaire application of knowledge. On the other hand, an extended
use of intuitive thought without attendant checks from analysis may be
equally unsatisfactory. Errors resulting from inappropriate use of intuitive
thought are not apt to be catastrophic, however, as errors resulting
from analytical thought are apt to be" [17, p
71].
The currently available methods of utility assessment are neither consistent
nor perfect, and there are good reasons to believe that they cannot be
made so [18, 19]. Though useful as
an adjunct to other, more traditional methods for value elicitation and
shared decision making, they are not a substitute. However, offering
to the clinician and patient a variety of methods that arrive at different
choices is by no means a tragic flaw. If used critically, these can be
an aid to living with uncertainty and sharing responsibility in clinical
practice.
In the long run, to evaluate utility assessment methods for use in clinical
practice we must keep in mind the impact that any proposed method has
upon the doctor-patient relationship [20]. A method
which enhances the trust and security in such a relationship should be
preferred not just because trust between doctor and patient is good in
itself, but also because it allows the ill patient to express preferences
that are based on his or her most mature values.
Acknowledgment
The authors wish to thank Drs. Barbara McNeil, Stephen Pauker, and other
members of the Society for Medical Decision Making, whose comments on
the initial presentation of this paper in 1979 were helpful in subsequent
revision. We should also like to thank Drs. Thomas Gutheil and Robert
Lawrence for providing the hospitable environment in which the final
revision took place.
References
-
McNeil BJ, Weichselbaum R, Pauker SG: Fallacy
of the five year survival in lung cancer. N Engl J Med 299 (25):
1397-1401, 1978
-
Raiffa H: Decision Analysis. Introductory
Lectures on Choices Under Uncertainty. Reading, Mass: Addison-Wesley,
1968
-
Poulton EC: The new psychophysics. Six models
for magnitude estimation. Psychol Bull 69:1-19, 1968
-
McNeil BJ, Weichselbaum R, Pauker SG: Speech
and survival. Tradeoffs between quality and quantity of life in laryngeal
cancer. N Engl J Med 305:982-987, 1981
-
Krzysztofowicz R, Duckstein L: Assessment
errors in multiattribute utility functions. Organ Behav Hum Perform
26:326-348, 1980
-
Farquhar PH, Fishburn PC: Equivalences and
continuity in multivalent preference structures. Working paper HBS
79-47. Division of Research, Harvard Business School, 1979, University
Microfilms International #80-12829
-
Hamm RM: The conditions of occurrence of the
preference reversal phenomenon. Ph.D. Dissertation, Harvard University,
1979
-
Lichtenstein S, Slovic P: Reversals of preference
between bids and choices in gambling decisions. J Exp Psychol 89:
46-55, 1971
-
Schoemaker PJH, Hershey JC, Kunreuther HC:
Biases in procedures for constructing utility functions. Paper presented
to Third Annual Meeting, Society for Medical Decision Making, Philadelphia,
October, 1981
-
Miyamoto J, Politser P: Risky utilities
and riskless values. A comparison of assessment procedures. Paper
presented to Third Annual Meeting, Society for Medical Decision Making,
Philadelphia, October, 1981
-
Quinn RJ: The effect of measurement method
on preference scales in a medical decision-making context. Paper
presented to Third Annual Meeting, Society for Medical Decision Making,
Philadelphia, October, 1981
-
Llewellyn-Thomas HA, Sutherland HJ, Tibshirani
RJ, et al: Health utility assessment using Von Neumann and Morgenstern’s
standard gamble. Fundamental assumptions. Paper presented to Third
Annual Meeting, Society for Medical Decision Making, Philadelphia,
October, 1981
-
Havens L: Taking a history from the difficult
patient. Lancet 1:138-140, 8
-
Bursztajn H, Hamm RM: Review of Diana Crane’s
The sanctity of social life. Physician’s treatment of critically
ill patients. Bioscience 29:112, 1979
-
Gutheil TG, Havens LL: The therapeutic alliances.
Contemporary meanings and confusions. Int Rev Psychoanal 6:467-481,
1979
-
Vaillant GE: Adaptation To Life. Boston:
Little Brown, 1977, pp 127-157
-
Hammond KR: Social judgment theory. Its
use in the study of psychoactive drugs. In, Hammond KR, Joyce CRB
(eds): Psychoactive Drugs and Social Judgment. Theory and Research.
New York: Wiley, 1975, pp 69-105
-
Tversky A, Kahneman D: The framing of decisions
and the psychology of choice. Science 211:453-458, 1981
-
Tversky A: Biases in preferences. Paper
presented to Third Annual Meeting, Society for Medical Decision Making,
Philadelphia, October, 1981
-
Bursztajn H, Feinbloom RI, Hamm RM, Brodsky
A: Medical Choices, Medical Chances. How Patients, Families and Physicians
Can Cope With Uncertainty. New York: Delacorte, 1981, pp 148-230