1. Statistics - deceptive? Authors often provide statistics to support their reasoning, and the statistics appear to be hard evidence. Authors often provide statistics to support their reasoning, and the statistics appear to be hard evidence. However, there are many ways that statistics can be misused However, there are many ways that statistics can be misused Because problematic statistics are used frequently, it is important to identify any problems with them Because problematic statistics are used frequently, it is important to identify any problems with them

2. Statistics – assessment clues 1 1. Try to find out as much as you can about how the statistics were obtained. Ask “How does the author know?” 2. Be curious about the type of average being described 3. Be alert to users of statistics concluding one thing, but proving another 4. Blind yourself to the author’s statistics and compared the needed statistical evidence with the statistics actually provided

3. Decision Making – do & recommend The thinking involved in deciding what to do, or in recommending a course of action, or weighing someone else’s recommendation, needs special attention since it is very common, is often important, and has to be evaluated in a distinctive way. The thinking involved in deciding what to do, or in recommending a course of action, or weighing someone else’s recommendation, needs special attention since it is very common, is often important, and has to be evaluated in a distinctive way. We are all used to making decisions, but sometimes we do that less skilfully than we might. We are all used to making decisions, but sometimes we do that less skilfully than we might.

4. Decision Making – personal & policy We are interested in decisions about substantial matters, such as personal ones, like which university to go to, what to study, whether to take a job, whether to become vegetarian. We are interested in decisions about substantial matters, such as personal ones, like which university to go to, what to study, whether to take a job, whether to become vegetarian. And also policy decisions, like whether cannabis should be legalized, boxing banned, fox-hunting with hounds made illegal, or whether parents should be punished when their children break the law, abortion permitted, and so on. And also policy decisions, like whether cannabis should be legalized, boxing banned, fox-hunting with hounds made illegal, or whether parents should be punished when their children break the law, abortion permitted, and so on.

5. Decision Making – react & deliberate Sometimes decisions have to be taken very quickly – perhaps to deal with an emergency – and then one just needs to act, or react. Sometimes decisions have to be taken very quickly – perhaps to deal with an emergency – and then one just needs to act, or react. But often we have do have time to think, or deliberate, and, if the decision matters, the question is how to make a good decision rather than a poor or a worse one. But often we have do have time to think, or deliberate, and, if the decision matters, the question is how to make a good decision rather than a poor or a worse one. Nevertheless, one should not think forever, and often a timed-out combination of deliberation and reaction is called for. Nevertheless, one should not think forever, and often a timed-out combination of deliberation and reaction is called for.

6. Decision Making – evaluate & satisfice It is common to say that the “optimum” is the enemy of the “good”. That striving for the optimum may end up consuming all our time and effort, without a decision being reached It is common to say that the “optimum” is the enemy of the “good”. That striving for the optimum may end up consuming all our time and effort, without a decision being reached Striving for the sufficiently good, simply satisficing, could have been conducive to an appropriate and timely decision and action Striving for the sufficiently good, simply satisficing, could have been conducive to an appropriate and timely decision and action Evaluation itself involves an expenditure, in time and resources, that must be taken into account as part a decision’s cost Evaluation itself involves an expenditure, in time and resources, that must be taken into account as part a decision’s cost

7. Decision Making – common flaws 1 Did not give the matter enough thought Did not give the matter enough thought Did the first thing we thought of Did the first thing we thought of Didn’t think of possible alternatives Didn’t think of possible alternatives Didn’t consider the consequences of various courses of action Didn’t consider the consequences of various courses of action

8. Decision Making – common flaws 2 Needed to get some more information Needed to get some more information Was too hasty Was too hasty Was too emotionally involved Was too emotionally involved Did what the ‘boss’ said Did what the ‘boss’ said Accepted without thinking what others suggested Accepted without thinking what others suggested

9. Decision Making – thinking map 1 1. What makes this decision Necessary? [Objectives?] 2. What is Recommended and on what Grounds? 3. What are the Options/Alternatives? Realistic or unusual? 4. What are the Possible Consequences of the various options – and How Likely are they? (On the basis of what evidence and how reliable is it?)

10. Decision Making – thinking map 2 5. How Important are these consequences – for all those affected? 6. When I Compare the alternatives in the light of the consequences, which is best? Is the recommended course best? 7. How can I carry out this decision? (Contingency plans?) 8. Is the decision taking too long and what is the cost of that? Is there a satisficing solution?

11. Decision Making – summary 1 The thinking involved in making decisions or in recommending a course of action deserves special attention. The thinking involved in making decisions or in recommending a course of action deserves special attention. We often do this poorly because we jump to a decision and afterwards rationalise it. We often do this poorly because we jump to a decision and afterwards rationalise it. To make decisions well and avoid the common weaknesses, we need to consider a reasonable range of options and their possible consequences before coming to a decision or an advice. To make decisions well and avoid the common weaknesses, we need to consider a reasonable range of options and their possible consequences before coming to a decision or an advice.

12. Decision Making – summary 2 Getting clear what the problem is may necessitate formulating objectives. Getting clear what the problem is may necessitate formulating objectives. In thinking of consequences we need to be as imaginative as possible, but also to judge carefully how likely they are, and how valuable, to come to a rational decision. In thinking of consequences we need to be as imaginative as possible, but also to judge carefully how likely they are, and how valuable, to come to a rational decision. To do this we may also need to undertake some investigations and take account of moral considerations. To do this we may also need to undertake some investigations and take account of moral considerations.

13. Omitted information - 1 By asking questions brought up in other sections, such as concerning ambiguity, assumptions, and evidence, we will detect much important missing information By asking questions brought up in other sections, such as concerning ambiguity, assumptions, and evidence, we will detect much important missing information A more complete search for omitted information, however, is so important to critical evaluation that it deserves additional emphasis A more complete search for omitted information, however, is so important to critical evaluation that it deserves additional emphasis Next we further sensitise to the importance of what is not said and remind that we react to an incomplete picture of an argument when we evaluate only the explicit parts Next we further sensitise to the importance of what is not said and remind that we react to an incomplete picture of an argument when we evaluate only the explicit parts

14. Omitted information - 2 Almost any information we encounter has a purpose. Its organization was selected and established by someone who hoped that it would affect our thinking in some designed way Almost any information we encounter has a purpose. Its organization was selected and established by someone who hoped that it would affect our thinking in some designed way Those trying to persuade us will almost always try to present their position in the strongest possible light Those trying to persuade us will almost always try to present their position in the strongest possible light It is wise to hesitate and think about what an author may not have told us, something our critical questioning has not yet revealed It is wise to hesitate and think about what an author may not have told us, something our critical questioning has not yet revealed

15. Omitted information - 3 Omitted information is inevitable, for at least five reasons: Omitted information is inevitable, for at least five reasons: 1. Time and space limitations 2. Limited attention span 3. Inadequacies inhuman knowledge 4. Deception 5. Existence of different perspectives

16. Clues for finding omitted information 1. Common counterarguments a. What reasons would someone who disagrees offer? b. Are there research studies that contradict the studies presented? c. Are there missing examples, testimonials, or analogies that support the other side of the argument? 2. Missing definitions: How would the arguments differ if key terms were defined in other ways?

17. Clues for finding omitted information 3. Missing value preferences or perspectives a. From what other set of values might one approach this issue? b. What kinds of arguments would be made by someone approaching the issue from a different set of values? 4. Origins of “facts” alluded to in the argument a. Where do the arguments come from? b. Are the factual claims supported by competent research or by reliable sources?

18. Clues for finding omitted information 5. Details of procedures used for gathering facts a. How many people completed the questionnaire? b. How were the survey questions worded? 6. Alternative techniques for gathering or organizing evidence: How might the results from an interview study differ from questionnaire results?