Monday, November 3 Evaluating research methods to (1) determine the strongest sides of scientific arguments and (2) generate content for the body of position papers IPHY 3700 Writing Process Map

Rhetorical Goal: Argue for the methodological strengths of studies that support your claim, to convince readers that the data derived from the studies are valid. Flip side: Argue for the methodological weaknesses of studies that do not support your claim, to convince readers that the data derived from the studies are problematic. 1. Clearly identify the most important and convincing methodological strengths of the studies that support your claim. For our paper I'm looking for deep holes, so you might focus on only 1 methodological strength from 1 supporting study. 2. Explain how the methodological strengths likely influenced the studies' outcomes, leading to valid results and conclusions. Avoid simply listing methodological strengths. 3. Explain how the strong methods in studies supporting your position are superior to weaker methods in studies supporting the counterarguments. That is, directly compare related methodological approaches across studies that support your argument and the counterargument.

Choosing Sides in Scientific Arguments Lane et al. Lane et al. Long-distance running, bone density, and osteoarthritis. Journal of the American Medical Association, 255, Marti et al. Marti et al. Is excessive running predictive of degenerative hip disease? British Medical Journal, 299, "These cross-sectional data do not suggest an increase in clinical osteoarthritis or in radiologic cartilage loss in runners, even when extreme running distances are involved. Running appears, in general from these data, not to predispose to development of osteoarthritis." "We think that long term, high intensity, high mileage running as performed by elite endurance athletes should not be dismissed as a potential risk factor or at least precipitating factor for premature osteoarthritis of the hip." How odd it is that anyone should not see that all observation must be for or against some view if it is to be of any service. —Charles Darwin (1861)

Process Activity: Evaluating research methods to choose sides and generate content 1. Throughout the process, stay focused on your rhetorical goals to guide your critique of research methods. 2. Create a summary table of research methods and results from studies on your issue. 3. Raise diagnostic questions for identifying strengths and weaknesses in research methods. 4. Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods. 5. Apply your evaluation of research methods to choose the strongest side of the argument and to generate content that helps you accomplish your rhetorical goals.

Create a summary table of research methods and results from the studies on your issue. StudySubjectsDesignIndependent Variables Dependent Variables Results Lane et al.Runners: n = 41; 18 women, 23 men; 57.5 years; 66.4 kg; 12,547 miles in 9.2 years Controls: n = 41; 18 women, 23 men; 57.7 years; 72.7 kg; 1,263 miles in 8.3 years *groups matched for education and occupation Cross-sectional: Subjects filled out a questionnaire (running history, diet, medical history) Subjects underwent a rheumatological exam: x-rays of the hands, lateral lumbar spine, and knee joints *hand x-rays served as controls Mean total miles run: 12,547 miles of running in 9.2 years for runners; 1,263 miles of running in 8.3 years for controls Index of severity of sclerosis and bone spurs; and joint space narrowing (mm) Sclerosis: Runners: 6.0 Controls: 5.3 Spurs: Runners: 7.5 Controls: 6.0 Narrowing: Runners: 7.0 Controls: 6.7 Marti et al.Runners: Members of Swiss National Team; n=27 men; mean age 27 at start of study; 73 mls/kg/min Controls: healthy untrained men; n=23; mean age 20 at start of study Longitudinal: over a 15-year period 1973: Subjects completed running and medical questionnaire 1988: Subjects completed running and medical questionnaire and were given x-ray of hips Mean total miles run: 97 km per week for runners Index of severity of sclerosis, bone spurs, joint space narrowing, and pain Reported as # of subjects affected Sclerosis: Runners: 16 Controls: 3 Spurs: Runners: 8 Controls: 0 Narrowing: Runners: 4 Controls: 0 Pain: Runners: 8 Controls: 0

Raise diagnostic questions for identifying strengths and weaknesses in research methods Some (But Not All!) Key Diagnostic Questions 1. How appropriate were the subjects' characteristics? Did the researchers screen the subjects appropriately? 2. Were subjects assigned to groups and conditions without bias? 3. Did the study include a sufficient number of subjects? 4. How appropriate was the study design? 5. How valid and comprehensive were the independent variables? 6. How valid, reliable, and comprehensive were the dependent variables? 7. How effectively did the researchers control for extraneous, or confounding, variables? 8. How fitting and accurate were the study's statistical tests? Handout: Evaluating Research Methods

Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods Diagnostic Question #1: How appropriate were the subjects' characteristics? Did the researchers screen the subjects appropriately? Diagnostic Question #2: Were subjects assigned to groups and conditions without bias? 1. The subjects should run a lot ( > 60 to 70 mi. per week) at a high intensity, because the physical impact of running on the joints could be a main factor that determines the risk for osteoarthritis. So if the subjects haven't reached the critical "extreme" dose of running (on which the researchers are basing their issue and argument!) they won't exhibit severe symptoms of osteoarthritis. So the researchers might falsely conclude that running doesn't increase the risk. 2. The subjects should include both men and women because the effects of long-distance running on the joints might differ across the sexes. The Q-angle tends to be greater in women than in men. A greater Q-angle might result in greater stress on the knee and hip joints in women runners (vs men runners). But another sex-specific factor involves estrogen. This hormone plays a role in preventing inflammatory diseases, such as the deterioration of cartilage. So, it's possible that premenapausal women runners will have a lower risk of developing osteoarthristis than men runners, who lack the protective effects of estrogen. 3. The subjects should be matched for body weight, because increased weight is a risk factor for developing OA. Here's a problematic design: In a cross-sectional study, let's say that the runners are lighter than the sedentary controls. The results show a lower incidence of OA markers in runners vs. controls. The researchers conclude that running reduces the risk of developing OA. But this conclusion is problematic because the lower incidence of OA in the runners might be due to their lighter weight rather than due to the running itself. Likewise, the greater incidence of OA markers in the control group might be due to their greater weight rather than to their sedentary lifestyle. Lane et al. Marti et al.

My Notes from Evaluating Lane et al.'s Methods Strengths 1. The researchers included both men and women as subjects; the percentage of men and women in each group (runners vs. controls) was similar. That's important because the effects of long-distance running on the risk for developing OA might differ across the sexes The subjects were matched for age. Increasing age is associated with increased risk for osteoarthritis. By matching runners and controls for age, the researchers avoided a situation in which one of the groups might have a higher incidence of OA markers due to their age rather than to their participation or lack of participation in long-distance running. 3. The subjects were matched for occupation. This control is important because.... Weaknesses 1. The subjects weren't matched for weight. The control subjects (mean = 72.7 kg) were significantly heavier than the runners (66.4 kg). Heavier individuals have a greater risk for developing OA. So it's possible that Lane's control subjects... you finish this "think- through"! And it's possible that Lane's runners... you finish this "think-through"! But see Lane et al.'s response to body weight as a confounding variable in the discussion section of their paper. 2. The runners don't represent the population to which the researchers are generalizing their conclusion! The runners do not cover "extreme" distances over long periods of time!!!! The subjects ran an average of only 26.2 mi. per week, or 3.7 mi. per day, over 9.2 years.

My Draft (208 words so far) A major flaw in Lane et al.'s study is that the subjects did not represent the target population to which the researchers generalize their results. Recall that Lane et al. concluded that running does not increase the risk of developing osteoarthritis, "even when extreme running distances are involved" (2, page 1151). Lane et al.'s subjects reported running an average of 12,547 miles over 9.2 years. Whereas that cumulative number of miles might seem impressive, it amounts to only 26.2 miles per week, or 3.7 miles per day. As you know, elite long-distance runners may average 60 to 100 miles (or more) per week over careers that can last 15 to 20 years. The relatively low volume of running reported by Lane et al.'s subjects might not have caused a criterion level of physical stress on the subjects' joints. If running increases the risk for osteoarthritis, it might do so through the repetitive stress it that places on the cartilage. (LSG: need to develop this section by explaining the mechanisms underlying a possible "dose-reponse" effect of running on OA) In contrast to Lane et al.'s subjects, Marti et al.'s subjects were not recreational runners; instead, they were members of the Swiss national track and field team. These subjects reported running an average of 97 kilometers (60.2 miles) per week....