Accredited Psychiatry & Medicine

Psychiatrists' Thorny Judgments
Describing and Improving Decision Making Processes

Robert M. Hamm, Ph.D.,
Jack Allen Clark, Ph.D.,
and Harold Bursztajn, M.D.

From the Center for Research on Judgment and Policy and the Department of Sociology, University of Colorado, Boulder, Colorado, and the Program in Psychiatry and the Law, Massachusetts Mental Health Center, Harvard Medical School, Boston, Massachusetts. Please address requests for reprints to Dr. Hamm, Center for Research on Judgment and Policy, Box 344, University of Colorado, Boulder, Colorado 80309, USA.

The decision making method used by Climo [1] for difficult decisions in public psychiatric hospitals is criticized for oversimplifying the decision situation. Two methods for describing decision processes, naturalistic observation and judgment policy analysis, are introduced as a basis for selecting techniques for clarifying and improving decision making in situations in which a formal decision analysis is not feasible. (Med Decis Making 4:425-447, 1984)

Introduction

In the preceding paper [1] Dr. Lawrence Climo has presented four examples of the difficult decision making task he faces in a public psychiatric hospital. He has described a "thorny" situation in which not only must the interests of patient, doctor, and society be considered, but in which these interests are also often unclear, ambiguous, or mutually opposed. This is a circumstance that strains not only rationality but also the imagination.

Climo's approach to analyzing his decision situation is to recognize competing interests, choose the most overriding one, and do so in a personally, professionally, and politically defensible manner. This approach is unsophisticated in comparison with the analyses usually seen in the pages of Medical Decision Making. Yet this is not his personal failing; for how would anyone apply decision analysis to such cases? The dilemmas in the public psychiatric hospital offer four hurdles to the decision analyst:

The situations are unique and so would need to be analyzed individually — a process that has been shown, in hospital settings for example, to take about two weeks' time [2] — hardly available to the harried staff of a public psychiatric hospital.
It is difficult to produce a decision tree for such a situation, i.e., to structure it in terms of available actions, possible events, and resulting outcomes. The situation is like a game against an opponent rather than a game against nature, and the opponent acts crazy or diabolical — i.e., is "divided against himself" [3] — and thus cannot be counted on to act in his or her own best interest. Hence, one cannot make reasonable simplifying assumptions that allow for the "pruning" of the decision tree. This is further complicated by the laws mandating that the sometimes incompatible rights of patient and society be protected and providing sanctions to the doctor if they are not.
Given a decision tree, it is difficult to measure the required probabilities. As the situations are unique, no data are available about the relative frequency of the possible events. And when the opponent is an erratic, unpredictable, even diabolical person, the probabilities of events are not independent of the actions the decision maker would take (see [4]).
Measurement of the utility of the anticipated outcomes is difficult. How does one evaluate an anticipated state of mental illness? It is difficult to get the psychiatric patient's reasonable cooperation in this evaluation process.

Enthusiasts no longer claim that decision analysis offers the solution to every kind of medical decision problem. Researchers and practitioners know how much hard work is involved in applying decision analysis to routine decision problems, let alone to intractable ones such as Dr. Climo's. Yet it would be wrong to conclude that, because a formal decision analysis is impossible, there is no hope for him to improve his decision making practice. As Raiffa [5] urges, the mere use of a decision analytic perspective may be of benefit. That is, to consider the decision in terms of actions, events, and outcomes, even if the probabilities of events and the utilities of outcomes cannot be rigorously measured, may make it more likely that the outcome will be good.

Even if formal decision analysis is not feasible, a psychiatrist faced with a difficult decision might have the following goals:

The decision process used should have a good chance of finding the right decision, within time, effort, and tool-availability constraints.
The decision process can be used consistently on a broad class of decisions.
The decision process can be defended, if the decision is criticized.
It should be possible to teach others to use the same decision process.
It should be possible to evaluate the decision process and improve it, if necessary.
It should be possible to involve the patient in the decision process.

To attain these goals, a number of steps will be required. Doctors' behavior in the decision situation should be observed very closely, and the results of these observations should be given to them. They should be shown that the use of concepts consistent with decision analysis can improve their decisions, and should be taught how to use these accurately. Finally, doctors should be taught how to recognize when techniques of various degrees of formality are feasible and appropriate.

In this paper we shall review and evaluate Climo's discussion of his thorny decisions. We will also present some suggestions for improving both his understanding of his difficult circumstances and his method for resolving the problems they cause him. Section I of the paper will provide perspective on Dr. Climo's decision process, Section I-A sketching the social nature of medical decision making and Section I-B presenting a framework for relating the kinds of reasoning decision makers use to the features of the decision situation; this framework will provide a basis for suggesting feasible techniques for making good decisions in these situations. In Section II, the cases Climo reports will be analyzed with respect to the kind of situation the decision presented and the kind of approach taken by the clinicians. In Section III, some ways outside researchers can help in clarifying the decision situation will be reviewed. In Section IV it will be shown how the psychiatrists' decision strategy may be improved. The particular role of the law in psychiatric decisions, and how the psychiatrist can use this kind of aided decision, is reviewed in the accompanying article by Bursztajn et al. [6].

I. Perspective on the Decision Process

I-A. Social Nature of Medical Decision Making

Climo's presentation of four cases and the thorny clinical judgments that they entail serves to remind us of the social character of medical decision making. In these cases, thorniness is a quality that inheres in the social context of his practice. Whereas from a strictly medical/psychiatric point of view, he may be faced with fairly straightforward problems of diagnosis and treatment, these problems are actually exceedingly difficult because of the troublesome social, political, and economic context in which he finds himself and his patients. This difficult context fosters the development of a professionally unconventional decision making method, designed to yield socially acceptable, if not medically valid, decisions.

The case reports illustrate two major aspects of the social character of medical decision making. First, they illustrate the significance of social factors in clinical judgment. In a sense, Dr. Climo has to take society as well as biomedical phenomena into account in deciding upon a diagnosis or a course of treatment. For instance, he clearly takes into account the interests of the court as well as those of his patient in deciding to renew Pepe's 20-day commitment. Second, the cases illustrate the socialized character of the decision making process itself. The cases show Dr. Climo making decisions through a process of social interaction with patients, judges, and fellow staff members. Thus not only does he necessarily take social factors into account in recognizing and evaluating alternatives and outcomes, but his cognitive process is also shared with others and thereby subject to control by social factors.

I-B. Locating Psychiatry's Dilemmas on the Cognitive Continuum

Many modes of reasoning may be observed in medical clinics, and although psychiatry is thought to involve particularly intuitive cognition, a range of approaches to reasoning may be seen there as well. At the same time, the use of a large number of decision aiding methods has been advocated, ranging from Dr. Climo's decision method to formal decision analysis. Hammond's Cognitive Continuum Theory [7,8] offers a basis for selecting such aids according to the kind of cognition prevailing in the situation. The theory characterizes cognition as being relatively intuitive or analytical and explains how the relative analyticity of cognition is partially determined by the task the person is thinking about. This framework will be useful in two ways. The perspective it provides can help us know in what situations a formal decision analysis would be inappropriate, and its descriptions of the kinds of cognition that are feasible in various situations can provide guidance in searching for methods for improving decision making. For example, it provides a framework for the development of the repertoire of utility assessment techniques and justifications for their selection, as called for by Bursztajn and Hamm [9].

Cognitive Continuum Theory [7,8] holds that cognition can vary along a continuum from intuition to analysis, and its position on this continuum is induced by the characteristics of the task. Both cognitive mode and task can be described by lists of characteristic features [7,8]; for example, intuition is rapid and analysis slow; intuition tends to combine cues with a linear weighted-average organizing principle while analysis tends to be more complicated, etc. Indices have been produced for measuring how analytical cognition is and how analysis-inducing tasks are [10].

Closely related to the descriptions of the tasks and the cognition they induce is a description of the modes of scientific reasoning that are characteristic of various positions on the cognitive continuum. Hammond [11] has described six modes, from the reasoning in the physics laboratory (Mode 1, most analytical) to the reasoning of a scientist making public policy outside his or her area of expertise (Mode 6, most intuitive). These six modes vary in a number of features, such as the covertness of the reasoning and whether the focus is on individual cases or on variables and relations among variables.

Two grand themes of medical reasoning are the problems of cognitive safeguards (How can we be sure our knowledge is correct?) and of correct action (How can we be sure our actions are correct?). The cognitive safeguards problem has been addressed [12,13] through descriptions of how people ensure that their thinking is correct in each of the six modes of cognition. Five features that distinguish among the modes have been identified [12]. Similarly, to look at the problem of correct action from this perspective, we can describe six distinct approaches to determining the proper action to take, ordered along the analytical/intuitive continuum (see Table 1).

Table 1. Features Characteristic of Each Mode on the Cognitive Continuum


Feature	Mode

	Analytical					Intuitive

	1	2	3	4	5	6
Decision structure, probabilities of events, and evaluations of outcomes are based on formal, well-established theary.	Yes	No	No	No	No	No
Probabilities and evaluations are based on controlled measurement of events and outcomes, at the least.	Yes	Yes	No	No	No	No
Measurements of event probabilities and outcomes are based on statistical summaries of large amounts of data, at least.	Yes	Yes	Yes	No	No	No
Decision model is generated and probabilities and evaluations of outcomes are measured, subjectively, at least.	Yes	Yes	Yes	Yes	No	No
Actions are justified with reference to reasons, rules, and principles, at least.	Yes	Yes	Yes	Yes	Yes	No

Modes 1 to 4 would involve the use of a formal decision analytical framework, including measurement of the probabilities and utilities. These modes differ in the analyticity of the methods used for measuring the probabilities of events and the goodness of outcomes. The kinds of decision analysis usually presented in the pages of Medical Decision Making are at Modes 3 and 4. When the probabilities are measured with reference to the relative frequencies of events in a large data base and the utilities are measured by systematically surveying a large number of people, it is a typical Mode 3 approach. When the probabilities and utilities are subjectively estimated, it is a typical Mode 4 approach.

The decision making reasoning typical of Mode 5 would not involve the explicit measurement of goodness or probability nor the calculation of expected utilities. Rather, it would involve public discussion of the decision with respect to principles. When decisions are made without an attempt at justification, e.g., on the authority of the doctor's "experience," it is likely that the most intuitive cognition, Mode 6, is involved.

In his paper, Climo has presented case descriptions where it is plain that the decisions are made with Mode 5 cognition: reasons for the decisions are discussed but there is no measurement. In seeking to improve the decision making in such cases (see Section IV), it is necessary to consider whether to try to improve the decision process at Mode 5, say by inducing the decision makers to use the concepts of actions, events, and outcomes, or to move the process up to Mode 4 by introducing measurement of probabilities of events and of the goodness of outcomes. Cognitive Continuum Theory plays a further role in improving the doctors' reasoning, in that its detailed descriptions of the various modes of cognition will enable us to select techniques that are compatible with the prevailing mode of cognition.

II. The Decision Making Practice of Dr. Climo and his Staff

Our critique of the style of decision making typical in psychiatric dilemmas must start with Climo's description of his and his staff's practice. He has described for us a common method that they use, and he has given us case examples. We shall evaluate this method, both as stated and as used in the cases.

II-A. The Reported Method

Climo [1] holds that it is reasonable and good for the psychiatric decision making team, in order to attain a happy solution to a difficult situation, to "take into account what ordinary, well-meaning, and responsible citizens might consider the right thing to do." He proposes that the method for doing this has two steps.

To consider which of the competing interests that have been identified feels, professionally and personally, the more overriding;
To consider which course of action would feel the least defensible if the staff were called to public accounting after the fact.

We note that this procedure calls on the reactions the doctors have as ordinary citizens as well as on their evaluations as psychiatrists. This may well be appropriate, unless it somehow would force them to do worse than they could, e.g., to oversimplify the case so that the ordinary citizen could understand and approve of it from a newspaper account [1].

We shall evaluate Climo's proposed decision making method in the normative terms of decision theory, asking, "If this procedure were generally used, would the decisions it produced be good?" We shall show, first, that the method is technically flawed because it could produce incoherent recommendations and, second, that the method is substantively flawed because it does not deal adequately with uncertainty or with the many aspects in a situation that should be considered.

II-B. Technical Objection: The Method Could Produce Incoherent Decisions

When one routinely faces unique cases under conditions of uncertainty, one may never have the data to evaluate whether a strategy is good or whether an alternative strategy would lead to better outcomes in the long run. One must evaluate the strategy by asking, instead, whether it is consistent or coherent.

The method that Climo finds to be common for dealing with these difficult decisions, and that he himself uses and recommends, is to identify the overriding interest and pick the action that is most defensible (or avoid the action that is least defensible) on this one overriding interest. It is not clear whether "interest" means the interests of different parties (e.g., patient, doctor, or society) or just different aspects of the outcomes, regardless of whose benefit is involved — most likely, the latter. For example, in the case of Pepe, the competing "interests" might be his discomfort on the ward and his possible benefit from the sound diagnosis that might be produced if he were kept on the ward for further observation. The second step of Climo's rule, that the most defensible action should be taken, seems to represent the doctor's interests; this implies that the first step is intended to address the different attributes of the situation. Either way, the decision rule can be shown to be incoherent.

The rule is incoherent because it recommends simplifying the problem so that only one attribute (the "overriding interest") is considered; the action is then chosen on the basis of that one attribute. The procedure for choosing which attribute is "overriding" is intuitive, based on "professional and personal feelings" in the situation. Tversky [14] has demonstrated that this kind of decision rule is capable of creating intransitive preferences, over a set of possible options. For example, imagine that the psychiatric hospital has only one spare bed, and you have to choose which of two patients to admit. Say that patient B is suffering more than patient A, while patient A has a slightly higher chance of committing suicide than patient B. You feel that the hospital can be effective both in alleviating the suffering and in preventing a suicide. When you compare the two patients, the difference in suicide risk may seem less tangible than the difference in suffering, and so you would elect to admit patient B. Next, you compare patient B with patient C, who has a slightly smaller chance of suicide than B but is suffering slightly more. By the same reasoning, you would now admit patient C. Similarly, comparing D with C, you might admit D. But if you now compare patient A with patient D, and D is suffering three steps more than A but A presents a three-steps-higher chance of committing suicide, the larger suicide risk of patient A might now look quite substantial, and you might admit patient A rather than patient D. These preferences are intransitive, for you admit patient B over (>) patient A, C > B, D > C, but A > D. And if you were willing to release a patient in order to admit one in greater need, you might find yourself back where you started, with patient A occupying the bed after much unnecessary trouble.

The mechanism by which Climo's decision rule would produce such an incoherence is that the first step, the judgment about which attribute is most important, is made in the context of the options that are available. One basis for deciding that an attribute is unimportant is that the difference between the available options on that attribute are too small to be perceived [15] or to be cared about [16]. However, as our example and Tversky's [14] empirical demonstrations showed, if the method were used on a (particular) series of options, the differences that were individually too small to attend to might sum to a difference that seemed to demand serious attention.

One would rarely encounter a set of psychiatric decisions that would so plainly demonstrate this incoherence in Climo's decision rule. In fact, the potential of such a flaw would not in itself prevent the method from being a good heuristic one, a method that works well in most situations. The possibility of intransitive decisions, however, which depend on the vagaries of the doctor's informal judgments of what attribute is most important, means that Dr. Climo cannot claim a special legitimacy for his decisions due to their having been produced by the method.

II-C. Substantive Objection: The Method Systematically Ignores Information

The fact that information is systematically ignored by the decision rule must be criticized on practical or substantive grounds. In principle, most decisions have many relevant attributes that should be taken into account, and most are made under conditions of uncertainty. Familiar arguments [17] hold that methods that attend to all the relevant aspects and to the uncertainties can potentially produce optimal decisions (especially if there is valid measurement of the uncertainties of events and of the utilities of outcomes). Conversely, the chance of producing the best decision is unavoidably lessened if one uses a method that systematically ignores aspects of the decision situation and/or the uncertainties. This is true even when one can justify using the method by saying truthfully that society demands a decision that can be explained simply or that the decision is too thorny for one to handle in its full complexity.

It is ironic that, according to Hammond's Cognitive Continuum Theory, doctors using intuitive cognition should be particularly good at considering a decision problem that has many aspects. The unconscious method by which intuitive cognition combines multiple attributes of a situation in evaluations or judgments of it is homologous to a "weighted linear average" in which each of the attributes is attended to with different "weights" or amounts of attention [17,18]. Although the weights may be wrong, at least the doctor using unaided intuition is likely to be attending to more of the factors than the doctor attempting to simplify the problem by attending to only the overriding interest. In proposing a method that ignores some factors, Climo may unintentionally be promoting worse decisions than if unaided intuition were used.

II-D. Opportunity for Bias

A further problem with systematically ignoring certain aspects of the decision situation is that the choice of which aspect to ignore may be made according to one of (at least) two institutionalized biases: the bias to act defensively and the bias to seek information.

The Bias to Act Defensively. Of course it is essential that the doctor act in such a way as to defend against the possibility of making negligent mistakes and suffering the consequences. But this defensiveness has costs. For example, it contributes to the overuse of diagnostic tests. To use a method that overemphasizes some motives and downplays others, when this defensiveness is one of the more powerful motives, is likely to increase rather than decrease the problems associated with the defensive practice of medicine.

The Bias to Seek Information. The case of Pepe presents an example in which great importance is placed on obtaining an exact diagnosis. In deciding what to do with the teenager who had been placed on a locked ward for a 20-day observation period, yet about whom no definite diagnosis had been reached, the staff considered:

The potential gain to the courts and to Pepe of keeping him for an additional observation period; a correct diagnosis could be of great benefit.
The costs to Pepe of the confinement; he found it very unpleasant.
The importance to the staff of discharging their institutional obligation to the court and to society by helping the court know what to do in this case.

It was decided that the first dimension, the value of the information that could be gained by keeping Pepe for another 20 days, was the overriding interest; but he killed himself after 5 of those days, perhaps motivated by the second dimension. We simply do not know whether the outcome would have been any different had Pepe been released. Nonetheless, oversimplification of the attribute structure of the decision may have been critical in this situation, because once the one dimension was labelled "important" and the others "unimportant," the staff's attention to Pepe's discomfort and to the possibility of his suicide may have decreased. Basing the decision on all aspects of the case might have led to attention to all aspects of Pepe's continuing management.

The same motivation to gain information was active in the management of an infant who for unknown reasons was not thriving, a case described in Chapter 1 of Medical Choices, Medical Chances [19]. In both these cases the doctors seemed to pay insufficient attention to the fact that the observations needed to gain information had an effect on the patient. Although the information may well be more important than the patient's feelings, choosing to attend only to this most important dimension can lead to a decision that is untempered by the necessary consideration of the patient's reaction to the situation.

II-E. Climo's Case Reports

Although Climo's method of identifying the overriding attribute and determining which actions would feel least defensible was applied in the case of Pepe, it does not seem to have been used in all of the cases he reports. For example, there is no mention of it in the report about Mr. Warren, the non-communicative psychotic who would not eat hospital food. One can criticize the handling of this case, not so much for a failure to attend to all attributes, but for the failure to deal explicitly with uncertainty.

In order to get the court to give the hospital control over Mr. Warren's body, it was necessary to prevent his cousin from feeding him, until he looked enough like an emergency for the court to give the hospital staff the desired control. Once they had this control, they were able to force him to eat and to take antipsychotic medicines, which brought about the anticipated improvement in his state. (Credit is due Dr. Climo and his team for inventing this option.) In thus trying to "shoot the moon" they were taking a gamble. If they won, they would do well on every relevant attribute: improved eating and lessened psychosis. If they lost, their situation would be little different from their starting place (unless the patient starved to death). A formal decision analysis might well reveal that the chosen option not only had higher expected value, but stochastically dominated any other option. However, lacking such an analysis, we would be more comfortable with the decision process in this case if we knew that Dr. Climo and his staff had acknowledged in their discussion that they were taking a gamble and had considered the possible bad outcomes and the likelihood of their occurrence. Although the problem might have been considered in these terms, there is no mention of it in Climo's narrative.

II-F. Insufficiently Detailed Information in Climo's Account

In discussing both Pepe's and Mr. Warren's cases, we have had insufficient information to criticize or praise the staff's decision process with complete confidence. In the case of Mr. Warren, for example, we would need to know more detail about whether the doctors recognized their strategy to be a gamble and, if so, how they evaluated it. Lacking such information, our evaluation cannot be fair to Dr. Climo and his staff, because we do not know exactly how the decision was made. A quote from the paper can illustrate the problem. Describing how the staff decided to keep Pepe on the locked ward for the second 20-day observation period, Climo writes, "All this was explained to Pepe and more time was requested" [1, p 420]. This sentence is the standard gloss that is entered into the patient's medical record to demonstrate that the patient's informed consent was obtained. But it tells us nothing about how the ideas were communicated, about what strategy was used to obtain Pepe's cooperation, about whether he agreed that the additional observation period was for his good, or about whether he was made to feel that the courts or the doctor or neither was on his side [20]. We need to know more about how these decisions are made before we can make specific suggestions about improving the decision making practices of Dr. Climo and his staff.

III. Methods for Describing the Decision Process

We have reached the limit of what we can say about the particular decision making processes used by Dr. Climo and his staff, given the inevitable loss of details that occurred in the production of his narrative account. But in order to say how we could help them — that is, how the decision making of any psychiatric team in a public mental hospital might be improved — we would need to know more about their decision making processes. In this section, we will describe two kinds of research methods that would produce the needed information. The first method, naturalistic observation, focuses on how decisions are made in individual cases by a group of people. As such, it is useful for studying decisions when the reasoning processes are relatively intuitive, say at Mode 5 or 6, given Hammond's [11] suggestion that a focus on cases is characteristic of these modes. The second method, judgment policy analysis, studies individuals' policies for judging a set of cases. It focuses on "variables" and hence is appropriate for judging clinical reasoning at the relatively analytic Mode 4.

The descriptions produced by these methods will be useful for three purposes:

Criticism: a detailed, accurate description of the decision process allows it to be evaluated, to determine whether it needs to be improved.
Cognitive feedback: the decision makers can be given feedback, in the form of a description of their decision making process, so that they can act with more self-awareness and evaluate the process themselves.
Assessment: knowing how well the doctors function at each mode helps us predict which attempts to give them useful cognitive tools or decision aids are likely to succeed.

III-A. Naturalistic Observation

First we shall describe procedures of naturalistic observation, derived from the ethnographic approach of sociology and anthropology. These analyze concrete cases as they occur in the life of the clinic [21,22].

The ethnographic method attends to the social nature of clinical decision making. As Climo's paper indicates, the problems that he faces are essentially social; they are parts of the social situation in which he practices. Moreover, the recognition and evaluation of alternatives and outcomes is carried out through social interaction. The unit of analysis is not a person, such as a clinician, but a case: the problematic matter ostensibly presented by a patient and requiring a decision by the clinician and his cohort. The analysis would be concerned with identifying the structure of social relations impinging on a given case and describing the dynamics of staff members' interactions as they present and evaluate alternatives and outcomes. It would reveal how a decision is embedded in a context of roles and socially distributed knowledge, perspectives, and interests.

The four case reports reveal staff meetings to be the principal settings of clinical decision making. Presumably, the alternatives and outcomes that Climo and his fellow staff members take into account are presented as topics of group discussion, and the process of evaluating and selecting alternatives is realized through conversational means. He refers to this collective mode of decision making when he says, in presenting the case of Pepe, "staff felt unready to make a conclusive diagnostic statement or offer any definite treatment recommendation" [1, p 420]. The staff meeting therefore provides an obvious starting point for an ethnographic analysis of clinical decision making. We might begin with an analysis of the content and process of social interaction that constitutes these dialogues.

Data collection for the analysis of social interaction in staff meetings entails the creation of two complementary sets of records [23]. One set is composed of complete records of the talk and other nonverbal aspects of the interaction that constitute a meeting, namely video and/or audio tape recordings and complete transcripts made from these recordings. These records would enable a detailed analysis of staff members' use of language and other paralinguistic cues, as well as an analysis of the structure and substance of their interaction.

Whereas tapes and transcript materials provide the principal data of social interaction in staff meetings, the analysis of these materials would be guided by information contained in a second data set. The second set would be produced through in-depth interviews of the various participants in the staff meetings. The participants would be treated as informants who provide their own understandings of the staff meeting. Such a procedure has been demonstrated [24], showing how the structure and effectiveness of doctor-patient communication can be assessed by having doctors and patients independently review, describe, and explain video tapes of their interactions.

The interviews of the participants would also attend to their respective professional and clinical backgrounds, their formal and informal relationships as staff members, and their shared background as collaborative clinicians in the hospital setting. The relevance of such data has been shown in studies of staff meetings in a residential care facility for emotionally disturbed children [25]. What participants said in staff meetings as they collectively constructed images of patients and negotiated the meaning of particular facts of a case was related to professional background (e.g., psychologist vs. social worker) and the organizational context in which each encountered the child (e.g., an examination room vs. a classroom).

The analysis of these materials would be concerned with examining two basic components of the discourse: the substance of the discourse and the processes of interaction through which it is carried out. The substantive component is further divided into twin concerns: what the staff members say to each other and what they talk about with each other.

The first entails attending to language and the details of linguistic performance, such as lexical choice, intonation, pitch, and rhythm. The analysis of language could attend to the presence/absence of certain professional or local-hospital symbols of psychiatric description ("buzz words"), or it could involve looking for usage indicative of decision analysis, including its formal vocabulary.

The second substantive concern is with what the staff members talk about. Language codes a culturally shared reality. Thus, while we attend to what the members say, their codes, we must also attend to what they talk about: the set of topics and meanings coded in the talk. Analysis of topics involves looking through the talk and inferring the underlying substantive meaning. This analysis relies on information gathered from the members' own reconstructive reflections on the discourse. This inferential analysis is particularly relevant in light of the fact that decision alternatives, outcomes, and utilities may be talked about and understood by the members without being explicitly presented as such. Collective decision-making may be carried out in a largely tacit manner.

The process component of analysis also consists of two subcomponents. One is the basic, "micro" processes of conversational interaction, including the processes involved in allocating turns to speak or in the coordinated accomplishment of question-answer couplets (see [26,27]). The other subcomponent consists of such "macro" processes as social role enactment and the exercise of power and authority. The first pertains to processes at work in the unfolding, utterance-to-utterance order of the discourse; the second pertains to collections of speech acts and actions that display professional and organizational role responsibilities.

The two levels of social process are integrated. For instance, a social control tactic may be realized through a subtle control of turn allocations. Analyses of conversational interaction have shown that people who ask questions are able to select who speaks next while maintaining the right to speak subsequent to the other's answer [26]. One person may seek to direct the construction of a set of alternatives according to his own professional concerns. His control of the agenda of a group discussion could be realized through a carefully orchestrated series of linked questions that lead to a certain conclusion while he maintains control of the floor. Trial lawyers are trained to be adept at such skills, while many physicians also display considerable expertise, especially in their dealings with patients (see the studies compiled by Fisher and Todd [28]).

The analysis of interaction within a staff meeting also calls for an extended analysis of the social relations outside it. The matters that are taken up, and the interests that are pursued, are linked to social relationships that lie beyond the particular time and space of the meeting. The findings, hunches, and expectations that staff members may present in reaching a diagnosis in a particular case are tied to the respective members' interactions with the patient. Conversely, external relationships are informed by norms and sets of relevances that are shared in the staff meeting. For example, interviewing routines that staff members adopt in their interactions with patients may be informed by their case-reporting responsibilities in staff meetings.

The staff meeting is therefore the centerpiece of a more comprehensive analysis of decision making realized in the social interactions of various parties in a complex social setting. Climo indicates that the meeting is the locus of his decision making, and that is where we might start in trying to reconstruct his thorny situation and his decision making method. We can look inside this event to see the interaction and shared decision making it contains. But our reconstruction of the situation and the decision making method also involves a reconstruction of the wider social setting itself, including the social relations that either are directly involved in or indirectly impinge on the meeting. The scope of the inquiry is difficult to predict. If we are to understand a particular case, however, we might begin with the meeting, move to include relevant doctor-patient and doctor-judge relationships, and then, as Climo suggested, take into account the historical relationship of the hospital to various other social institutions and to the policy at large.

III-B. Judgment Policy Analysis

The second method for observing the decision process is derived from the judgment and decision research tradition of psychology. It focuses on variables; that is, it develops a description of the doctor's decision policy in terms of how the decision depends on the attributes of the situation. Instead of describing how the doctor considers the concrete details of a single situation, this method describes his or her decision policy at a more abstract level, in terms of how variations in the key attributes would affect the decision. The description has these elements:

A set of attributes, dimensions, or cues that are important in the decision.
A method of measuring each of these variables.
A description of how the variables are aggregated or organized to arrive at a decision.

The aggregation process descriptions typically assume that the model has a "weighted average" organizing principle. In studying clinical decision making, this assumption is justified by (a) Hammond's [7,8] theory that intuition involves such a weighted averaging process (note that his theory explicitly opposes the oft-stated assumption that intuition involves configural combination of cues) in combination with (b) our assumption that the doctors are operating at Mode 5 or 6, the intuitive end of the cognitive continuum. The aggregation process can then be represented with a model of the form

D =	Σ_i W_i X_i*
	Σ_i W_i

where the decision variable D is a weighted average of the impacts of the dimensions, and each dimension i is measured on scale X_i and multiplied by weight W_i.

We shall describe three procedures that can be used to find numbers for such a model. These vary in difficulty and also in the accuracy of the parameters they produce.

Self-reports. The first procedure is to ask doctors to report the relative weights they use, using a convenient scale (e.g., Edward's "SMART" procedure [29]). Although this is simple for the researcher to administer and takes little of the doctors' time, to do it accurately would require great sophistication and abstract reasoning on the doctors' part. People's self-reports of weights are not particularly precise [30], although the decrement in predictive accuracy may not be severe [31]. However, self-report accuracy would be particularly difficult to attain for decisions that are usually handled intuitively, where the doctors do not routinely talk about the situations in terms of variables. The procedure is useful, however, for getting doctors to think qualitatively about the relative importance of different aspects of the decision situation without completely ignoring the less important dimensions, and it thus represents an advance over Climo's reported decision method.

Selective Focus on Variables. The second procedure is to present a detailed description of a decision situation and to vary each attribute in turn over its entire possible range, to determine the effect of this variation on the doctors' evaluation. (This is similar to the procedures of Keeney [32] and Gabrielli and von Winterfeldt [33]; see also Meehl [34]).

Compared to the first procedure, this procedure takes more of the doctors' time and requires mathematical sophistication on the part of the researcher. On the other hand, accuracy at this task would not in theory require as much sophistication on the doctors' part. Stewart and Ely [35], however, have demonstrated a flaw in a similar procedure: people's reported weights are insensitive to changes in the range the dimension is varied over. Therefore, though demanding less sophistication from the doctors, this procedure still probably requires reasoning that is too abstract for them to deal with accurately. Consequently, the numbers derived from this procedure should not be trusted; specifically, they should not be used in a policy that stands in the doctor's stead. Nonetheless, this procedure can make doctors take a careful, variable-oriented look at their decision situation.

Judging Hypothetical Cases. In the third procedure a set of hypothetical cases is constructed to vary systematically on the key attributes; the doctor judges each case, and statistical procedures relating his judgments to the cues are used to produce the best-fit parameters for the model. The set of attributes should include the relevant social and institutional factors, in addition to "clinical" variables. Determining the set may require the researcher to interview a number of doctors about what they pay attention to in these situations or to observe the decision process directly.

This procedure requires more work than the other two procedures. The doctor must judge each of a large number of cases. The cases are concrete, however, so no special sophistication is required of the doctor for accurate response. The researcher must construct the case materials and do the statistical analysis. The advantage of this procedure is that the resulting model represents the doctor's judgment policy accurately over the space of possible cases spanned by the set of hypothetical cases. (If one is concerned about the possibility of nonlinear cue use or nonlinear aggregation principles, one can use more elaborate statistics (e.g., [36,37]). A possible disadvantage of this method is that the hypothetical nature of the cases encourages the doctor to take too abstract an attitude toward them. Hence the judgment policy description may reflect the doctor's cool [38], even bored [39], reasoning, while if faced with real decisions the doctor might be passionately involved. Yet the judgment model is an accurate description of the doctor's cool cognition, and as such it may be useful for giving perspective in an upsetting case.

These three procedures are not equally useful for our purposes of criticism, cognitive feedback, and assessment. For criticizing the doctor's policy by comparing a descriptive model of it with a prescription, the third procedure is best, because its description is more accurate. When difficulty for doctor and researcher is an issue, however, the easier procedures may give a rough indication of problems with the policy.

For cognitive feedback, the doctors could learn little about their decision policies from the first procedure, since they supply the parameters directly. The third procedure would give more accurate feedback than the second.

For assessment of the doctor's ability to reason in terms of variables, any of the three procedures could be used. In addition to describing the doctor's policy, the third procedure measures his or her consistency; this is useful because a doctor who applies a policy consistently is probably able to profit from instructions and techniques, particularly if they are accompanied by cognitive feedback.

IV. Improving Decision Making in Thorny Psychiatric Situations

Determining the best approach for improving the decision making processes of psychiatrists faced with the kind of difficult decision that Climo has described may be conceived as a three-step process: observation, evaluation, and assessment.

IV-A. Observation of the Decision Process

The researchers must find out exactly how the decisions are made, using, for example, the methods reviewed in Section III, as appropriate. In addition, it will be useful to determine at what mode of cognition those who play a role in the decision are functioning. We recognize that the doctor is capable of thinking at each of a number of cognitive modes, and functions, if only at a rudimentary level, at most of these some of the time. The level of analyticity of his or her current cognitive activity is partially determined by a number of factors in the situation: by the kind of information that is available (accurate or not), by the kind of control available (amount of chance or uncontrollability), by the quality of reasoning that is expected of the doctor, or by the kinds of cognitive tools that are available and that the practitioners are trained, motivated, and institutionally supported to use.

IV-B. Evaluation of the Decision Process

The researchers must determine whether there is need for improvement, as we did with Climo's practice in Section II — except that it would be necessary to work with a more detailed description of the decision process. The process might be evaluated with respect to a variety of standards: for example, by comparison with the kinds of reasoning the law sanctions [6], with the norms of decision theory, with psychiatric standards pertaining to the therapeutic use of the decision process [20], with empirical standards of accuracy in judgment, or with outcome studies.

IV-C. Assessment of the Improvement Options

The available intervention options should be assessed in terms of their need, feasibility, and likely success. To determine the best mode of intervention, researchers need to answer the following questions.

What degree of guidance is appropriate? Should researchers simply give feedback, showing the decision makers descriptions of their decision processes? Or should we tell them how to change their practices in order to make their decisions better? This choice depends on whether it would be clear to the psychiatrists how to improve their practice once their behavior has been reflected to them. Some of the normative grounds against which we may measure their practice may be unfamiliar, and so the doctors would need guidance. In some cases, no improvement may be possible without the adoption of new practices in which the doctors would need extensive tutoring. On the other hand, in some situations simply reflecting the decision makers' behavior to them may reveal something that they know immediately how to correct. Here our choice might be whether to hold the mirror for them, or to let them hold it for themselves; perhaps they can be taught simple yet effective descriptive techniques that can be used independently of the researchers.
At what cognitive mode should the intervention be made, whether it be feedback or explicit guidance? We assume that most participants use reasoning from a range of modes on the cognitive continuum: their unjustified intuitive reactions (Mode 6), their publicly justified case-focused reasoning, such as referring to other similar cases (Mode 5), and variable-focused reasoning with or without attempts at measurement (Mode 4). We must determine at what level to target an intervention, taking into account the level of the current decision processes and whether the staff is ready to handle feedback or techniques at a more analytical level.

Feedback at the Mode 5 level of cognition would involve explaining the findings of the case-focused, descriptive analysis, as provided by the method described in Section III-A. As this type of analysis refers to a theory of how decisions are made as well as to a theory of how social interactions produce such decisions, the decision makers would surely receive much insight into their own decision processes. They could evaluate their processes and choose to act in a way they judge to be better, helped perhaps by the researcher's continued monitoring.

Feedback at Mode 4 would involve explaining the findings of the variable-focused descriptive methodology of Section III-B, for example, or of a description of their behavior in terms of subjective expected utility theory [40]. Use of such feedback, in combination with a convincing argument that the person's judgments should make different use of the variables (e.g., that it would be better to put different weights on the attributes of the decision situation [41] or that the decision maker's choices violate the axioms of decision theory [42]), has proven effective at inducing people to change their judgment strategies.

Active intervention at the Mode 5 level of cognition would involve teaching the decision makers to use vocabulary that makes explicit the considerations that decision theory would require them to attend to: the concepts of options, events, and outcomes, and the idea of measuring and comparing the uncertainty of events and the value of outcomes (see [19]). The use of such concepts, even in the absence of explicit measurement of uncertainties or values, probably contributes to good decision making [5].

Active intervention at Mode 4 could involve teaching techniques for aiding judgment or inference, using some form of subjective measurement of variables. For example, the decision makers could be taught to use decision analysis with subjective probability and utility assessments [43], nonstatistical analytical techniques [44], an empirically derived rule for choosing the best therapy [45], or "bootstrapping," which is the use of a model, derived from their own judgments, that captures the essence of their judgment policy and discards the noise due to their inconsistent use of the policy, thus producing better judgments than they make on their own [46].
Which techniques should be used with which members of the decision making team? Different people, with different abilities, will bear the main responsibility for different parts of the decision process. We must take into account which part of the decision process might bring the most gain, if it were to be improved. But we must also consider the ability, motivation, training, and role expectations of the person who is responsible for that part of the decision process. Further, these considerations are interdependent: one would not wish to teach one capable and motivated person to measure probabilities in a sophisticated manner if it were not possible to measure utilities, or if the people in a position to assess utilities were not willing to do so. Finally, one person or a coalition may have the power to determine the details of the decision process. Giving others feedback or teaching others new techniques may be wasted unless this power holder supports the interventions and the required changes in the decision process.

Just as with the introduction of specific techniques, in choosing the general mode of cognition at which to intervene with a group of decision makers it is important to be sensitive to what is feasible in the context of the entire decision making team. One should not automatically push for the most analytical of the available techniques. Even if some participants are capable and motivated to learn Mode 4 techniques, one must anticipate how this will fit in with the whole decision process, that is, how well the techniques will be supported by existing or possibly reorganized social relations in the group. Getting all staff members to use the vocabulary of decision theory in an informal way might produce more improvement and less resistance than teaching a few members to use sophisticated techniques and expecting the group to use the information they produce.

V. Conclusion

We have criticized the decision making practice of Dr. Climo and his staff, as described in his paper, explained two types of method for describing decision processes in detail, and outlined how to choose among a number of general approaches for helping doctors faced with thorny psychiatric decisions. How can the methods and techniques we have reviewed help a psychiatrist attain the goals listed in the introduction of this paper?

An appropriate, principled use of decision theory, whether this be the use of its vocabulary at Mode 5 or of its measurement techniques at Mode 4, would increase the likelihood of finding the best decision.
Guiding one's practice by a decision policy derived by using the descriptive methods of Section III-B would increase the consistency of the decision process over the class of decisions that the policy or model applies to.
If the decision process uses decision theoretic concepts accurately and appropriately, these offer a powerful justification, should the decision be criticized.
An accurate description of the decision making process, whether produced by case-focused (Section III-A) or variable-focused (Section III-B) description techniques, can be used to teach others how to use it.
Evaluation of the decision process and proposed changes in it is feasible when descriptions are available for comparison with normative theories.
The vocabulary of rational decision making can be used for communicating both with patients and among staff members, and for teaching patients. In order not to confuse and alienate patients, however, it might be better to explain to most patients at the level of Mode 5 rather than Mode 4.

Thus the ideas we have reviewed for describing and improving the decision making process in the public psychiatric hospital can contribute to the attainment of each of the goals a psychiatrist would have for the decision making process.

Progress in this area requires the participation of practitioners as well as researchers. We thank Dr. Climo for taking the risk of explaining his difficult situation and his method for dealing with it. We hope that our work will stimulate practitioners to study their decision processes and seek methods for improving them, and stimulate researchers to develop relatively intuitive techniques that promote practice consistent with decision theory in real, thorny situations.

References

Climo LH: Some thorny medical judgments and their outcomes. The view from the public mental hospital. Med Decis Making 4:415-424, 1984
Zarin DA, Plante DA, Kassirer JP, Pauker SG: Experience with a consultative service in clinical decision making. Paper presented to the Fifth Annual Meeting of the Society for Medical Decision Making, Toronto, October 1983. Abstract in Med Decis Making 3:371, 1983
Buie DH: Empathy. Its nature and limitations. J Am Psychoanal Assoc 29:281-307, 1981
Bar-Hillel M, Margalit A: Newcomb's paradox revisited. Br J Philosoph Sci 23:295-304, 1972
Raiffa H: Theories of decision making and medical practice. Keynote Address to the Fourth Annual Meeting of the Society for Medical Decision Making, Boston, October 1982
Bursztajn H, Hamm RM, Gutheil TG, Brodsky A: The decision-analytic approach to medical malpractice law. Formal proposals and informal syntheses. Med Decis Making 4:401-414, 1984
Hammond KR: The integration of research in judgment and decision theory. Report No. 226. Boulder CO: Center for Research on Judgment and Policy, University of Colorado, 1980
Hammond KR: Principles of organization in intuitive and analytical cognition. Report No. 231. Boulder CO: Center for Research on Judgment and Policy, University of Colorado, 1981
Bursztajn H, Hamm RM: The clinical utility of utility assessment. Med Decis Making 2:161-165, 1982
Hammond KR, Hamm RM, Grassia J, Pearson T: The relative efficacy of intuitive and analytical cognition. A second direct comparison. Report No. 252. Boulder CO: Center for Research on Judgment and Policy, University of Colorado, 1984
Hammond KR: Toward increasing competence of thought in public policy formation. In, Hammond KR, ed: Judgment and Decision in Public Policy Formation. Boulder CO: Westview, 1978, pp 11-32
Hammond KR, Hamm RM, Fisch H-U, Joyce CRB: Differential contributions of two forms of cognitive science to medical decision making. Paper presented to the Fourth Annual Meeting of the Society for Medical Decision Making, Boston, October 1982. Abstract in Med Decis Making 2:358, 1982
Hammond KR, Hamm RM: Thoughts on the acquisition and application of medical knowledge. In, Friedman CP, Purcell EF, eds: The New Biology and Medical Education. Merging the Biological, Information, and Cognitive Sciences. New York: Josiah Macy, Jr., Foundation, 1983, pp 190-197
Tversky A: Intransitivity of preferences. Psychol Rev 76:31-48, 1969
Luce RD: Semiorders and a theory of utility discrimination. Econometrica 24:146-159, 1956
Ng Y-K: Sub-semiorder. A model of multidimensional choice with preference intransitivity. J Math Psychol 16:51-59, 1977
Raiffa H: Decision Analysis. Introductory Lectures on Choices Under Uncertainty. Reading MA: Addison-Wesley, 1968
Shanteau J: The concept of weight in judgment and decision making. A review and some unifying proposals. Report No. 228. Boulder CO: Center for Research on Judgment and Policy, University of Colorado, 1980
Bursztajn H, Feinbloom RI, Hamm RM, Brodsky A: Medical Choices, Medical Chances. How Patients, Families and Physicians Can Cope with Uncertainty. New York: Delacorte, 1981
Gutheil TG, Bursztajn H, Brodsky A: Malpractice prevention through the sharing of uncertainty. Informed consent and the therapeutic alliance. N Engl J Med 311:49-51, 1984
Light D: Becoming Psychiatrists. New York: Norton, 1982
Lidz C, et al: Informed Consent. A Study of Decision Making in Psychiatry. New York: Guilford Press, 1984
Clark J: The conversational art of diagnosis. The social construction of medical facts. PhD dissertation, University of Colorado, Boulder, 1984
Frankel RM, Beckman HB: IMPACT. An interaction-based method for preserving and analyzing clinical transactions. In, Pettegrew LS, ed: Explorations in Provider and Patient Interaction. Louisville, KY: Humana, Inc., 1982
Buckholdt DR, Gubrium JF: Doing staffings. Hum Organ 38:255-264, 1979
Sacks H, Schegloff E, Jefferson G: A simplest systematics for the organization of turn-taking for conversation. Language 50:696-735, 1974
Sudnow D: Studies in Social Interaction. New York: Free Press, 1972
Fisher S, Todd AD, eds: The Social Organization of Doctor-Patient Communication. Washington: Center for Applied Linguistics, 1983
Edwards W: Use of multiattribute utility measurement for social decision making. In, Bell DE, Keeney RL, Raiffa H, eds: Conflicting Objectives in Decision. New York: Wiley, 1977, pp 247-276
Slovic P, Lichtenstein S: Comparison of Bayesian and regression approaches to the study of information processing in judgment. Organ Behav Hum Perform 6:649-744, 1971
Cook RL, Stewart TR: A comparison of seven methods for obtaining subjective description of judgmental policy. Organ Behav Human Perform 13:31-45, 1975
Keeney RL: The art of assessing multiattribute utility functions. Organ Behav Human Perform 19:267-310, 1977
Gabrielli WF, von Winterfeldt D: Are importance weights sensitive to the range of alternatives in multiattribute utility measurement? Research Report 78-6. Los Angeles: Social Science Research Institute, University of Southern California, 1978
Meehl P: Specific etiology and other forms of strong influence. Some quantitative meanings. J Philos 2:33-53, 1977
Stewart TR, Ely DW: Range sensitivity. A necessary condition and a test for the validity of weights. Boulder Colorado: National Center for Atmospheric Research, 1984
Anderson NH: Methods of Information Integration Theory. New York: Academic, 1982
Hammond KR, McClelland GH, Mumpower J: Human Judgment and Decision Making. Theories, Methods, and Procedures. New York: Praeger, 1980
Janis IL, Mann L: Decision Making. A Psychological Analysis of Conflict, Choice, and Commitment. New York: Free Press, 1977, pp 45-202
Slovic P, Lichtenstein S, Edwards W: Boredom-induced changes in preferences among bets. Am J Psychol 78:208-217, 1965
Eisenberg JM, Hershey JC: Derived thresholds. Determining the diagnostic probabilities at which clinicians initiate testing and treatment. Med Decis Making 3:155-168, 1983
Hammond KR, Summers DA, Deane DH: Negative effects of outcome-feedback in multiple-cue probability learning. Organ Behav Hum Perform 9:30-34, 1973
Slovic P, Tversky A: Who accepts Savage's axiom? Behav Sci 19:368-373, 1974
Weinstein MC, Fineberg HV, Elstein AS, et al: Clinical Decision Analysis. Philadelphia: Saunders, 1980, pp 168-227
Spiegelhalter DJ: Statistical aids in clinical decision-making. Statistician 31:19-36, 1982
Centor RM, Witherspoon JM: Treating sore throats in the emergency room. The importance of follow-up in decision making. Med Decis Making 2:463-469, 1982
Dawes RM, Corrigan B: Linear models in decision making. Psychol Bull 81:95-106, 1974

Psychiatrists' Thorny Judgments Describing and Improving Decision Making Processes