generated from rstudio/bookdown-demo
-
Notifications
You must be signed in to change notification settings - Fork 0
/
01-judgment.html
371 lines (324 loc) · 21.3 KB
/
01-judgment.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
<!DOCTYPE html>
<html lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>01-judgment.knit</title>
<meta name="description" content="" />
<meta name="generator" content="bookdown 0.36 and GitBook 2.6.7" />
<meta property="og:title" content="01-judgment.knit" />
<meta property="og:type" content="book" />
<meta name="twitter:card" content="summary" />
<meta name="twitter:title" content="01-judgment.knit" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black" />
<script src="libs/jquery-3.6.0/jquery-3.6.0.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/fuse.min.js"></script>
<link href="libs/gitbook-2.6.7/css/style.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-table.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-bookdown.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-highlight.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-search.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-fontsettings.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-clipboard.css" rel="stylesheet" />
<link href="libs/anchor-sections-1.1.0/anchor-sections.css" rel="stylesheet" />
<link href="libs/anchor-sections-1.1.0/anchor-sections-hash.css" rel="stylesheet" />
<script src="libs/anchor-sections-1.1.0/anchor-sections.js"></script>
<style type="text/css">
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
</style>
<link rel="stylesheet" href="style.css" type="text/css" />
</head>
<body>
<div class="book without-animation with-summary font-size-2 font-family-1" data-basepath=".">
<div class="book-summary">
<nav role="navigation">
<ul class="summary">
<li><a href="./">Table of Contents</a></li>
<li class="divider"></li>
<li class="chapter" data-level="1" data-path=""><a href="#systems-of-human-judgment"><i class="fa fa-check"></i><b>1</b> Systems of human judgment</a>
<ul>
<li class="chapter" data-level="1.1" data-path=""><a href="#system-1-and-system-2-further-comments"><i class="fa fa-check"></i><b>1.1</b> System 1 and System 2 — further comments</a></li>
<li class="chapter" data-level="1.2" data-path=""><a href="#the-intuition-of-professionals"><i class="fa fa-check"></i><b>1.2</b> The Intuition of Professionals</a>
<ul>
<li class="chapter" data-level="" data-path=""><a href="#obstacles-to-effective-judgment"><i class="fa fa-check"></i>Obstacles to effective judgment</a></li>
</ul></li>
<li class="chapter" data-level="1.3" data-path=""><a href="#a-demand-for-discipline-careful-thought"><i class="fa fa-check"></i><b>1.3</b> A demand for discipline & careful thought</a></li>
<li class="chapter" data-level="1.4" data-path=""><a href="#further-examples"><i class="fa fa-check"></i><b>1.4</b> Further examples</a>
<ul>
<li class="chapter" data-level="" data-path=""><a href="#the-conjunction-fallacy"><i class="fa fa-check"></i>The “conjunction fallacy”</a></li>
<li class="chapter" data-level="" data-path=""><a href="#even-careful-critics-sometimes-get-it-wrong"><i class="fa fa-check"></i>Even careful critics sometimes get it wrong</a></li>
<li class="chapter" data-level="" data-path=""><a href="#think-again-a-very-simple-example"><i class="fa fa-check"></i>Think again — a very simple example</a></li>
<li class="chapter" data-level="" data-path=""><a href="#a-test-to-check-understanding-of-risk"><i class="fa fa-check"></i>A test to check understanding of risk</a></li>
</ul></li>
<li class="chapter" data-level="1.5" data-path=""><a href="#misbehaving-humans"><i class="fa fa-check"></i><b>1.5</b> Misbehaving humans!</a></li>
<li class="chapter" data-level="1.6" data-path=""><a href="#negotiating-life-in-an-uncertain-world01-judgment-5"><i class="fa fa-check"></i><b>1.6</b> Negotiating Life in an Uncertain World</a></li>
</ul></li>
<li class="divider"></li>
<li><a href="https://github.com/dataThoughts" target="blank">Published with bookdown</a></li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i><a href="./"></a>
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<section class="normal" id="section-">
<!--bookdown:title:end-->
<!--bookdown:title:start-->
<div id="systems-of-human-judgment" class="section level1 hasAnchor" number="1">
<h1><span class="header-section-number">Chapter 1</span> Systems of human judgment<a href="#systems-of-human-judgment" class="anchor-section" aria-label="Anchor link to header"></a></h1>
<p>An understanding of the mental traps to which we are prone can help us avoid them. Kahneman’s book <span class="citation">[-@kahneman_2013]</span> <em>Thinking Fast and Slow</em> is a good starting point for thinking about the strengths and limitations of human thinking processes. There is no good substitute for the use of “educating gossip”, as Kahneman describes it, for training in effective judgment and in decision making.</p>
<p>Important themes that Kahneman notes are</p>
<ul>
<li>We have “an excessive confidence in what we think we know”.</li>
<li>We too readily judge decisions by outcome, rather than by how they are made.</li>
<li>We have two selves — an experiencing self and a remembering self
<ul>
<li>These do not always have the same interests.</li>
</ul></li>
<li>Automatic memory formation has its own rules
<ul>
<li>This is exploited to improve the memory of a bad episode</li>
</ul></li>
<li>“We easily think associatively, … metaphorically, casually, but statistics requires thinking about many things at once …”</li>
</ul>
<div id="system-1-and-system-2-further-comments" class="section level2 hasAnchor" number="1.1">
<h2><span class="header-section-number">1.1</span> System 1 and System 2 — further comments<a href="#system-1-and-system-2-further-comments" class="anchor-section" aria-label="Anchor link to header"></a></h2>
<p>Humans have been conditioned to respond quickly to immediate risks and challenges, without stopping to consider too carefully whether what we heard was a false alarm. They also have the ability, when the occasion seems to demand it, to stop to ponder. This is the basis for Kahneman’s categorization of human thought processes as of two types — System 1 which jumps rapidly to make a judgment, and System 2 which takes time for careful
consideration.<a href="#fn1" class="footnote-ref" id="fnref1"><sup>1</sup></a></p>
<p>System 1 Features are</p>
<ul>
<li>It may answer an easier question in place of a harder.</li>
<li>It responds to irrelevancies — priming, framing, affect, memory illusions, illusions of truth, …</li>
<li>It has little understanding of logic and statistics</li>
<li>It cannot be turned off</li>
<li>When flummoxed, it calls on System 2</li>
</ul>
<p>System 2 features are</p>
<ul>
<li>It keeps you polite when angry, alert when driving at night</li>
<li>In its world, gorillas do not cross basket-ball courts …<a href="#fn2" class="footnote-ref" id="fnref2"><sup>2</sup></a></li>
<li>There are some vital tasks that only System 2 can perform</li>
<li>Problems that put 1 & 2 in conflict may require large mental effort & self-control to overcome the impulses and intuitions of System 1.</li>
</ul>
<p>Both systems are amenable to training. A well-trained System 2 helps
greatly in creating a better System 1. Further points are:</p>
<ul>
<li>Healthy living is a compromise
<ul>
<li>Recognize situations where mistakes are likely</li>
<li>Aim to avoid significant mistakes when the stakes are high</li>
</ul></li>
<li>Untrained humans are poor intuitive statisticians</li>
<li>Judgments about statistical issues may require us to think about more than one, even many, things at once.</li>
<li>Too often, we jump to conclusions, without careful assessment.</li>
<li>We may not be equipped to make an informed and carefully thought through decision.</li>
</ul>
</div>
<div id="the-intuition-of-professionals" class="section level2 hasAnchor" number="1.2">
<h2><span class="header-section-number">1.2</span> The Intuition of Professionals<a href="#the-intuition-of-professionals" class="anchor-section" aria-label="Anchor link to header"></a></h2>
<p>Effective professional training is designed to ensure that at least some of the results of well-tuned System 2 expert judgment operate at a System 1 level.
The professional will, if the training is doing its job, build up a
repetoire of System 2 judgments that will later, when the circumstances
seem to demand it, be available at a System 1 level.</p>
<blockquote>
<p>The situation has provided a cue; this cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition.<br />
<span class="citation">[@simon1992explanation, "What is an Explanation of Behavior?"]</span></p>
</blockquote>
<div id="obstacles-to-effective-judgment" class="section level3 unnumbered hasAnchor">
<h3>Obstacles to effective judgment<a href="#obstacles-to-effective-judgment" class="anchor-section" aria-label="Anchor link to header"></a></h3>
<p>Even those who are experts in their field can be similarly prone to judgments that have no foundation in fact:</p>
<blockquote>
<p>“But you’re not an MD like I am. The problem with you is that you do not understand medicine. You see, medical statistics are not like other statistics.”<br />
<span class="citation">[Prostate biopsy specialist, in response to evidence that screening may be counterproductive, quoted in @levitin_2015, p.247]</span></p>
</blockquote>
<p>The following comment appeared in a discussion of the response to a U.S. Preventive Services assessment that prostate screening, when used in accordance with then current treatment practices, was doing more harm than good.<a href="#fn3" class="footnote-ref" id="fnref3"><sup>3</sup></a></p>
<blockquote>
<p>Even faced with … evidence … from a ten-year study of around 250,000
men that showed the test didn’t save lives, many activists and medical professionals are clamoring for men to continue receiving their annual PSA test.</p>
</blockquote>
<p>New evidence emerges as time proceeds, and there are advances in the approach to treatment. At least part of the problem has been a rush to treatments that themselves risk increasing damage and the risk of death. Note the comment in <span class="citation">@brawley2018prostate</span> that</p>
<blockquote>
<p>Over the past few years, the benefit‐to‐harm ratio has improved in favor of benefit if the man understands that active surveillance may be a reasonable path if diagnosed.</p>
</blockquote>
</div>
</div>
<div id="a-demand-for-discipline-careful-thought" class="section level2 hasAnchor" number="1.3">
<h2><span class="header-section-number">1.3</span> A demand for discipline & careful thought<a href="#a-demand-for-discipline-careful-thought" class="anchor-section" aria-label="Anchor link to header"></a></h2>
<ul>
<li>We make judgments based on evidence that is too limited</li>
<li>We are easily fooled by irrelevancies
<ul>
<li>Kahneman has brought together evidence on what & how.</li>
</ul></li>
<li>Even when data are there for the taking, someone has to notice, to collate the data, and to understand its uses and limitations</li>
<li>Randomized controlled experiments (RCTs) are often the ideal, but require meticulous planning. If effects of interest are small, the numbers required may be very large. See further, Subsection <a href="#ss:rct"><strong>??</strong></a>
<ul>
<li>A limitation is that results apply only to the population from which
trial participants were taken. Any wider generalization has to be justified</li>
</ul></li>
<li>Observational data does not easily, if at all, substitute for the use of RCTs. It is no easy matter to adjust for sources of bias</li>
<li>Keep in mind Yule-Simpson “paradox”, which we will encounter later in Subsection <a href="#sec:yule1"><strong>??</strong></a>
<ul>
<li>The paradox lies in the failure of human intuition to accommodate straightforward arithmetic!</li>
</ul></li>
</ul>
</div>
<div id="further-examples" class="section level2 hasAnchor" number="1.4">
<h2><span class="header-section-number">1.4</span> Further examples<a href="#further-examples" class="anchor-section" aria-label="Anchor link to header"></a></h2>
<div id="the-conjunction-fallacy" class="section level3 unnumbered hasAnchor">
<h3>The “conjunction fallacy”<a href="#the-conjunction-fallacy" class="anchor-section" aria-label="Anchor link to header"></a></h3>
<p>This has also, as a result of the example given in
<span class="citation">@tversky1983extensional</span> come to be known as “the Linda problem”.
The name ‘Linda’ comes from the question and usual response
that are given by way of example.</p>
<blockquote>
<p>Linda is a 31-year old philosophy graduate, single, outspoken, and bright. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which of the following is more probable?</p>
</blockquote>
<ul>
<li>Linda is a bank teller.</li>
<li>Linda is a bank teller, and active in the feminist movement.</li>
</ul>
<p>Adding the further descriptor “active in the feminist movement” can only lower the probability, or just possibly leave it unchanged. Instead of assessing the balance of probabilities, we are tempted to ask which description best meshes with what we have been already told about Linda.</p>
<p>“Linda is active in the feminist movement” is the single descriptor” that respondents see as best fitting Linda. While that was not what was asked, one has to pay close attention to prevent System 1 from substituting that for the question that was asked. Note that the correct answer will be “a bank teller”, irrespective of the way that Linda was characterized before the question was asked.</p>
<p>In part, the issue is one of use of language. The “correct” answer is asking us to use the word “probable” in a strict technical sense.</p>
</div>
<div id="even-careful-critics-sometimes-get-it-wrong" class="section level3 unnumbered hasAnchor">
<h3>Even careful critics sometimes get it wrong<a href="#even-careful-critics-sometimes-get-it-wrong" class="anchor-section" aria-label="Anchor link to header"></a></h3>
<p>An irony is that Kahneman was, as he has acknowledged, himself fooled into taking at face values papers that claimed to show that verbal concepts could have the effect of altering behaviour. Thus</p>
<ul>
<li>Being asked to write down stories about unethical deeds made people more likely to want to buy soap;</li>
<li>Subtly drawing attention to money, e.g., leave banknotes lying around, made people feel more self-sufficient, and care less about others;</li>
<li>Priming people with old age related words leads people to walk more slowly away from the lab as research assistants armed with stopwatches timed their movements.</li>
</ul>
<p>As <span class="citation">@ritchie2020science</span> notes (p.28) Kahneman was not alone in being fooled — the study about priming with old age related words has been extensively cited in psychology textbooks. None of these claims have stood up in attempts at replication, with larger numbers and with greater care to avoid unconscious sources of bias. Thus, in the replication of the study relating to age-related words, infra-red beams were used to measure time taken to walk between two points in a hallway, rather than research assistants who knew the group to which participants had been assigned.</p>
</div>
<div id="think-again-a-very-simple-example" class="section level3 unnumbered hasAnchor">
<h3>Think again — a very simple example<a href="#think-again-a-very-simple-example" class="anchor-section" aria-label="Anchor link to header"></a></h3>
<p>Is symptom X associated with disease A?</p>
<table>
<thead>
<tr class="header">
<th></th>
<th>Has Disease</th>
<th>No Disease</th>
<th></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>Symptom X Present</td>
<td>20</td>
<td>10</td>
<td></td>
</tr>
<tr class="even">
<td>Symptom X Absent</td>
<td>80</td>
<td>40</td>
<td></td>
</tr>
</tbody>
</table>
<p>The symptom occurs with the same relative frequency, whether or not a person has the disease. Nisbett comments that most people, including nurses and doctors, interpret such evidence wrongly <span class="citation">[@nisbett, pp.129-130]</span>.</p>
</div>
<div id="a-test-to-check-understanding-of-risk" class="section level3 unnumbered hasAnchor">
<h3>A test to check understanding of risk<a href="#a-test-to-check-understanding-of-risk" class="anchor-section" aria-label="Anchor link to header"></a></h3>
<p>See the 2-minute test <a href="http://www.riskliteracy.org/">“Do you understand risk?”</a><a href="#fn4" class="footnote-ref" id="fnref4"><sup>4</sup></a></p>
</div>
</div>
<div id="misbehaving-humans" class="section level2 hasAnchor" number="1.5">
<h2><span class="header-section-number">1.5</span> Misbehaving humans!<a href="#misbehaving-humans" class="anchor-section" aria-label="Anchor link to header"></a></h2>
<p>The discipline of “behavioural economics” largely took shape as a result of the work of Richard Thaler. Kahneman was one of two mentors who strongly influenced Thaler — the other was Amos Tversky, who had worked closely with Kahneman.</p>
<p><span class="citation">@thaler2015misbehaving</span> explores the extent to which humans do not behave like the rational agents of classical economics, agents to whom Thaler gives the name “econs”. Added to the irrationality with which we often act is that our personal priorities are unlikely to align precisely with those of econs.</p>
<p>Note also comments in Part 4 of <span class="citation">@kahneman_2013</span>, titled “A conversation with the discipline of economics”. In a discussion on “The Prospect theory model of choice”, Kahneman comments on</p>
<ul>
<li>“… unfortunate tendency to treat problems in isolation”</li>
<li>Framing effects — inconsequential features shape choices</li>
</ul>
<p>Hence, “a challenge to the assumptions of standard economics”.</p>
</div>
<div id="negotiating-life-in-an-uncertain-world01-judgment-5" class="section level2 hasAnchor" number="1.6">
<h2><span class="header-section-number">1.6</span> Negotiating Life in an Uncertain World<a href="#fn5" class="footnote-ref" id="fnref5"><sup>5</sup></a><a href="#negotiating-life-in-an-uncertain-world01-judgment-5" class="anchor-section" aria-label="Anchor link to header"></a></h2>
<p>Questions that it can be helpful to ask include</p>
<ol style="list-style-type: decimal">
<li>Risk of what? (Showing a symptom, …, Death)</li>
<li>What is the time frame? (next 10 years, or lifetime)</li>
<li>How big is the risk? (Look at risk in absolute terms)</li>
<li>Does the risk apply to me? (Age, sex, health, …)</li>
<li>What are the harms of “finding out”? (False alarms, invasive diagnostic procedures, unnecessary or dangerous treatments.)</li>
</ol>
</div>
</div>
<div class="footnotes footnotes-end-of-document">
<hr />
<ol>
<li id="fn1"><p>^01-judgment-1<a href="#fnref1" class="footnote-back">↩︎</a></p></li>
<li id="fn2"><p>See <a href="http://theinvisiblegorilla.com/gorilla_experiment.html" class="uri">http://theinvisiblegorilla.com/gorilla_experiment.html</a><a href="#fnref2" class="footnote-back">↩︎</a></p></li>
<li id="fn3"><p>Association of Professional Psychologists, web post on <span class="citation">@arkes2012psychological</span><a href="#fnref3" class="footnote-back">↩︎</a></p></li>
<li id="fn4"><p><a href="http://www.riskliteracy.org/" class="uri">http://www.riskliteracy.org/</a><a href="#fnref4" class="footnote-back">↩︎</a></p></li>
<li id="fn5"><p>These questions came originally from the Harding Center web site <a href="https://www.hardingcenter.de/en" class="uri">https://www.hardingcenter.de/en</a>.<a href="#fnref5" class="footnote-back">↩︎</a></p></li>
</ol>
</div>
</section>
</div>
</div>
</div>
</div>
</div>
<script src="libs/gitbook-2.6.7/js/app.min.js"></script>
<script src="libs/gitbook-2.6.7/js/clipboard.min.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-search.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-sharing.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-fontsettings.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-bookdown.js"></script>
<script src="libs/gitbook-2.6.7/js/jquery.highlight.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-clipboard.js"></script>
<script>
gitbook.require(["gitbook"], function(gitbook) {
gitbook.start({
"sharing": {
"github": false,
"facebook": true,
"twitter": true,
"linkedin": false,
"weibo": false,
"instapaper": false,
"vk": false,
"whatsapp": false,
"all": ["facebook", "twitter", "linkedin", "weibo", "instapaper"]
},
"fontsettings": {
"theme": "white",
"family": "sans",
"size": 2
},
"edit": {
"link": "https://github.com/jhmaindonald/dataThoughts/master/%s",
"text": "Edit"
},
"history": {
"link": null,
"text": null
},
"view": {
"link": null,
"text": null
},
"download": null,
"search": false,
"toc": {
"collapse": "subsection"
}
});
});
</script>
</body>
</html>