The Clinical Utility of Utility Assessment

Harold Bursztajn, M.D., and Robert M. Hamm, Ph.D.

From the Program in Psychiatry and the Law, Massachusetts Mental Health Center, Boston, Massachusetts, and the Center for Research on Judgment and Policy, University of Colorado, Boulder, Colorado. Dr. Bursztajn’s participation in this work was supported in part by NIMH Grant #TOTMH64603. Dr. Hamm was a research fellow in the Division of Primary Care and Family Medicine, Harvard Medical School.

An earlier version of this paper was presented to the First Annual Meeting of the Society for Medical Decision Making, Cincinnati, Ohio, September 12, 1979. Please address requests for reprints to Dr. Bursztajn, Program in Psychiatry and the Law, Massachusetts Mental Health Center, 74 Fenwood Road, Boston, Massachusetts 02115, USA.

Abstract

The usefulness of utility assessment as a method for revealing individual patients’ desires is limited by two methodological problems. Different utility assessment methods can yield inconsistent results, both within a single clinical context and in different contexts; and the methods may not reflect universal rules that a patient may wish to abide by. To make utility assessment useful to the clinician, future research needs to address these problems in the clinical context. (Med Decis Making 2:161-165, 1982)

Article

A variety of current trends express the need for greater patient participation in decision making. An increasingly influential response to this need is to use utility assessment procedures to measure individual patients’ preferences for the outcomes of possible treatments. For example, McNeil, Weichselbaum, and Pauker [1] used a utility assessment procedure to measure individual patients’ preferences for duration of life, indicating which of two treatments for lung cancer would be preferred -radiation or surgery. This is an important demonstration of the use of utility theory in medical decision making. However, before widespread use of such a utility measure as a means for ascertaining patient values would be justified, further questions about its reliability and validity need to be answered. The sorts of questions that one must ask can be clarified by examining the version of the lottery method of subjective utility assessment [2] that is commonly used, as in the case of the paper by McNeil et al. This method produces a &dquo;utility curve,&dquo; showing the relation between the amount of life remaining and the subjective value of that amount of life. We ask, first, whether it is wise to rely on only one such method to measure utility?

Poulton [3] has shown that when asking for subjective judgments of sensory magnitudes, the particular details of the methods can have major effects on the subjective relation revealed. For example, when judging the loudness of sounds, asking for judgments of stimuli near some limit, such as the threshold of hearing, may give a different overall relation than if such quiet stimuli are not considered, because the subjective impression of those stimuli is different near the threshold. Looking for analogous effects in the assessment of utility, we wonder whether a lottery method that uses a 50% chance of dying in one week as its zero point might not yield utility curves less risk averse than one that anchors on a 50% chance of "dying immediately" [1]. We note that in a later work McNeil, Weichselbaum, and Pauker [4] have sought to avoid this problem by offering 50-50 gambles between survival for one’s full life expectancy and death in a few months. Another method would be to partition the range of outcomes into subranges that the patient finds it natural to think about, assess the utility function over each range, and then combine the functions into one function for the whole range, in the manner of Krzysztofowicz and Duckstein [5] and Farquhar and Fishburn [6].

It has been shown that it makes a difference whether people indicate preference by taking a choice between two alternatives (as do McNeil, Weichselbaum, and Pauker’s patients) or by directly naming numbers to represent utility. The systematic inconsistencies found between the two methods [7, 8] give us reason to consider very carefully what methods we want to use. Recent work showing how different utility scales are produced by different lottery method procedures [9-12] supports this recommendation.

A second major problem is that it is not easy to bring all reasonably relevant considerations into discussions with patients. How may this be done in such a way as to guarantee the validity of the utilities of remaining life measured by a method such as a 50-50 lottery? First, we must expect that fear and denial [13] would be manifest in patients with severe illness of recent onset. These could make patients either more or less risk averse than they might be under other conditions. Such patients might not think of certain important considerations such as the impact that their health care choices might have on the well-being of others, including their families, loved ones, and the community as a whole. They might give a place of specious prominence to other, less important considerations. For example, one’s self-interest may come to be far more narrowly conceived, excluding altruism and concern for others, as well as being given a greater emphasis than it had prior to the procedure [14]. These effects can happen with either the lottery or the direct assessment method. To counter these, a method for clinical use should have some of the characteristics of a well-done psychiatric interview that, by providing a supportive relationship, seeks to free the patient from overwhelming fear and aloneness in the face of illness. The hope would be that the patient might thereby have freer access to those values which can be most easily expressed when one is secure enough to function at one’s most mature level [15, 16].

There are also certain preferences some patients have which may be quite difficult to express with these utility assessment methods. A patient may think that the decision in his or her case should be made according to universal considerations, such as those expressed in religion and philosophy, rather than according to the chaos of one’s momentary impulses. If one wants one’s behavior to be governed by universal rules, it may not be immediately clear how to express this using the lottery method. Suppose that a patient may have a notion that it is good to die quickly, cleanly, and with the dignity that is often attributed to operative death. It is not obvious to an untutored patient (or indeed to us) how to express these values in terms of the lottery method, when its only parameters are time of life left. Though it may be possible to express this universal rule through the lottery method, can we expect the patient to be able to do this well? Perhaps training in using the lottery method to express principles would solve this problem, or again a special procedure modeled on the psychiatric interview could be undertaken to elicit the patients’ principles and explore how these would be reflected in the utility assessment method.

It might well be possible to use the lottery method to express the preferences embodied in most universal rules. However, there are some universal rules that could militate against the use of the 50-50 lottery method itself. A rule against suicide, for example, might make it difficult for a patient to consider taking a gamble with a 50% chance of immediate death. This bias against a particular aspect of one of the alternatives would make the patient appear more risk averse than another method might reveal. Special procedures are needed to elicit the universal rules that patients may wish to use, so as to avoid the automatic application of assessment methods in a manner that undermines the results and defeats the purpose of the utility assessment.

In order for clinicians to be able to apply utility theory by using the results of work such as the study by McNeil et al. [1], it is necessary that the methods be embedded in a repertoire which offers a variety of methods and justifications for their selection. Why is one utility assessment procedure more appropriate than another? Justifications for selecting a particular method would depend on evidence that other available methods will not yield substantially different results, or if they do, there must be some basis for preferring either the method or the pattern of results that the chosen procedure produces.

The research necessary to create such a repertoire would involve comparing the results of the different methods in each of a variety of situations. The results of this comparison, embodied in the rules of thumb for selecting from the repertoire, could alert the user to possible pitfalls and systematic reversals of preference engendered by the use of particular methods in particular situations.

As a final caveat, a repertoire of values assessment methods would be incomplete if, when using experimental comparisons to construct it, the "method of no method," i.e., the clinician’s intuitive approach, were not considered. The clinician’s approach offers a valuable comparison, not just as a "control" to see whether these systematic utility assessment methods are any better than "no method at all," but because it would not have the systematic errors that can sometimes result from using a particular elicitation procedure. Such a critical balance between the systematic and the intuitive approaches to assessment of patients’ values is needed. "Professional students of society ... often make large errors of judgment because of an overly analytical and doctrinaire application of knowledge. On the other hand, an extended use of intuitive thought without attendant checks from analysis may be equally unsatisfactory. Errors resulting from inappropriate use of intuitive thought are not apt to be catastrophic, however, as errors resulting from analytical thought are apt to be" [17, p 71].

The currently available methods of utility assessment are neither consistent nor perfect, and there are good reasons to believe that they cannot be made so [18, 19]. Though useful as an adjunct to other, more traditional methods for value elicitation and shared decision making, they are not a substitute. However, offering to the clinician and patient a variety of methods that arrive at different choices is by no means a tragic flaw. If used critically, these can be an aid to living with uncertainty and sharing responsibility in clinical practice.

In the long run, to evaluate utility assessment methods for use in clinical practice we must keep in mind the impact that any proposed method has upon the doctor-patient relationship [20]. A method which enhances the trust and security in such a relationship should be preferred not just because trust between doctor and patient is good in itself, but also because it allows the ill patient to express preferences that are based on his or her most mature values.

Acknowledgment

The authors wish to thank Drs. Barbara McNeil, Stephen Pauker, and other members of the Society for Medical Decision Making, whose comments on the initial presentation of this paper in 1979 were helpful in subsequent revision. We should also like to thank Drs. Thomas Gutheil and Robert Lawrence for providing the hospitable environment in which the final revision took place.

References

  1. McNeil BJ, Weichselbaum R, Pauker SG: Fallacy of the five year survival in lung cancer. N Engl J Med 299 (25): 1397-1401, 1978
  2. Raiffa H: Decision Analysis. Introductory Lectures on Choices Under Uncertainty. Reading, Mass: Addison-Wesley, 1968
  3. Poulton EC: The new psychophysics. Six models for magnitude estimation. Psychol Bull 69:1-19, 1968
  4. McNeil BJ, Weichselbaum R, Pauker SG: Speech and survival. Tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med 305:982-987, 1981
  5. Krzysztofowicz R, Duckstein L: Assessment errors in multiattribute utility functions. Organ Behav Hum Perform 26:326-348, 1980
  6. Farquhar PH, Fishburn PC: Equivalences and continuity in multivalent preference structures. Working paper HBS 79-47. Division of Research, Harvard Business School, 1979, University Microfilms International #80-12829
  7. Hamm RM: The conditions of occurrence of the preference reversal phenomenon. Ph.D. Dissertation, Harvard University, 1979
  8. Lichtenstein S, Slovic P: Reversals of preference between bids and choices in gambling decisions. J Exp Psychol 89: 46-55, 1971
  9. Schoemaker PJH, Hershey JC, Kunreuther HC: Biases in procedures for constructing utility functions. Paper presented to Third Annual Meeting, Society for Medical Decision Making, Philadelphia, October, 1981
  10. Miyamoto J, Politser P: Risky utilities and riskless values. A comparison of assessment procedures. Paper presented to Third Annual Meeting, Society for Medical Decision Making, Philadelphia, October, 1981
  11. Quinn RJ: The effect of measurement method on preference scales in a medical decision-making context. Paper presented to Third Annual Meeting, Society for Medical Decision Making, Philadelphia, October, 1981
  12. Llewellyn-Thomas HA, Sutherland HJ, Tibshirani RJ, et al: Health utility assessment using Von Neumann and Morgenstern’s standard gamble. Fundamental assumptions. Paper presented to Third Annual Meeting, Society for Medical Decision Making, Philadelphia, October, 1981
  13. Havens L: Taking a history from the difficult patient. Lancet 1:138-140, 8
  14. Bursztajn H, Hamm RM: Review of Diana Crane’s The sanctity of social life. Physician’s treatment of critically ill patients. Bioscience 29:112, 1979
  15. Gutheil TG, Havens LL: The therapeutic alliances. Contemporary meanings and confusions. Int Rev Psychoanal 6:467-481, 1979
  16. Vaillant GE: Adaptation To Life. Boston: Little Brown, 1977, pp 127-157
  17. Hammond KR: Social judgment theory. Its use in the study of psychoactive drugs. In, Hammond KR, Joyce CRB (eds): Psychoactive Drugs and Social Judgment. Theory and Research. New York: Wiley, 1975, pp 69-105
  18. Tversky A, Kahneman D: The framing of decisions and the psychology of choice. Science 211:453-458, 1981
  19. Tversky A: Biases in preferences. Paper presented to Third Annual Meeting, Society for Medical Decision Making, Philadelphia, October, 1981
  20. Bursztajn H, Feinbloom RI, Hamm RM, Brodsky A: Medical Choices, Medical Chances. How Patients, Families and Physicians Can Cope With Uncertainty. New York: Delacorte, 1981, pp 148-230