Writing About Methods in Doctoral Capstone Studies Kelly Chermack, PhD, Dissertation Editor Walden University Writing Center Staff

Session overview & objectives In this presentation, we will cover: The contents of the Methods Section/Chapter General writing tips for writing about methods Writing about reliability, validity, and researcher bias Examples of well-written sections Links to methods writing resources

Questions and Recording Type in the Questions box Access captioning Download slides View recording http://writingcenter.waldenu.edu/415.htm Before we begin, let’s discuss how to interact with the material, both within the presentation and afterward. We’ll be stopping at various points to cover any questions that arise. If you have questions during the presentation, type them in the Questions box on your screen, and we’ll be sure to address them. If you want to access the captioning for this webinar, you can click the link in the Chat box. After the presentation, feel free to download the PowerPoint slides and view the recording. These will both be housed on our Webinars page in the archive.

Wait a second. Where do I even begin? How do I get from there to here?

Beginning to write… Step 1: Download the correct template and the appropriate rubric or checklist for your program. Step 2: Review the headings create an outline. Be sure to insert the necessary headings into your template. Step 3: Begin writing each the content for each heading.

Beginning to write… Remember, your research question determines your method. When beginning writing this chapter or section, you should already have determined your method. In this chapter/section, you should describe your research design and how you conducted your study, describe your sample, and your measures. See pages 29-32 in the APA Manual (6th Edition)

Commonalities among dissertations and docstudies: Clear introduction Identification of the methodology and research design The purpose of the study and the research questions Description of population and sample, sampling method, data collection processes The role of the researcher and informed consent (where applicable) Please see your specific rubric or checklist for more details on methods content (more on where to download these documents at the end of the presentation)

General APA writing tips Writing in APA style requires clear, concise, and detailed writing When writing about your methods, remember to reflect on the following: Is it clear to the reader what method I am using and why? Is it clear to the reader who was involved in my study? Population Sample Other personnel (from Walden or from your research site)

General APA writing tips (continued) Is it clear to the reader what my process was while: Obtaining access to my research site and sample Determining my sampling procedures and inviting respondents to participate For students at the final capstone stage: Did I include my Walden IRB approval number (in this chapter or section, or in an appendix)? Reinforce going back through chapters for changes for final draft.

Verb tense Use the correct verb tense to describe your project and data collection methods. In the proposal, use the future tense to describe methods and processes which you have not yet conducted Example: “Using a phenomenological approach, I will explore the effects of childhood obesity on self-esteem in young adults.” In the final capstone, use the past tense to describe research that has been completed Example: “Using a cross-sectional survey, I analyzed the effects of exposure to violence as a child on divorce rates in mid-life.”

Voice APA stresses that writers use the active voice. This means that the writer clearly indicates the actor or agent who is doing or taking the action in the sentence This is especially important while describing data collection processes Example: “In this study, data were collected using intensive interviews.” Instead: “In this study, I conducted intensive interviews.”

Voice Yes! It is acceptable to use the first person (“I”). In the methods chapter/section, however, a lot of I’s can be a bit much: Example: “In this study, I administered a survey. I created a convenience sample of 68 teachers. I invited them to participate in the survey by emailing them an invitation. I obtained email addresses from the principal of the school…”

Voice To the reader, it is redundant when each sentence begins with “I.” In order to avoid this, use “I” in the first sentence of the paragraph. Then, as long as it is clear that you are the actor in the remaining sentences, the passive voice is allowed: Instead: “In this study, I administered a survey. Using a convenience, sample of 68 teachers were invited to participate in the survey by emailing them an invitation. The principal of the school provided email addresses…”

Voice Also, avoid writing in the third person. Example: “The researcher introduced the study to participants and explained to them that they could refuse to answer any questions or stop the interview at any time.” Instead: “I introduced the study to participants and explained to them that they could refuse to answer any questions or stop the interview at any time.”

Anthropomorphism Anthropomorphism occurs when writers attribute human-like qualities to inanimate objects Example: “This study explored the link between leadership coaching and manager approval ratings.” Instead: “In this study, I explored the link between leadership coaching and manager approval ratings.”

Citations Knowing what to cite, and when, is tricky. Balance between: demonstrating an understanding of the method keeping the writing streamlined and thoughtful Although citing Creswell is helpful, also reference other authors outside of standard or textbook methods pieces Reference specific, exemplary articles on your method

Citations Remember, we cite authors when we want to credit something that they wrote. What if you want to cite an author or piece because it is a good example of the type of methodology you are using? In this case, you are not citing them because you want to give credit to a finding, but to give credit to their research as an example of the method Give credit to the author by using an “e.g.” in the parenthetical citation

Citations Example: “Since 1975, two longitudinal surveys have measured binge drinking (e.g., Johnston et al., 2010; Substance Abuse and Mental Health Services Administration, 2010).” In this case, these authors did not write that that two surveys had measured binge drinking, they are the two surveys that measured this construct

Citations Another example: “My methods follow previous studies (e.g., Archer, 1991; Epstein, 2008; Oransky & Marecek, 2009; Witt, 1997), that determined that gender-role development influences behavior and self-perception.” In this example, the writer is citing previous studies with similar findings, not authors who claimed that their research supported previous findings

Questions?

The next few slides will help you write about reliability, validity, and your role as a researcher

Writing about reliability Distinguish between whether you are reporting on the already-calculated reliability of a standardized instrument or on the reliability of an instrument that you have created or somehow modified. Include: The name of the scale and the number of items of the subscale The type of reliability test you ran The reliability coefficient (e.g., Cronbach’s alpha or Pearson’s r) Optional, but recommended: Comment on the quality of that reliability coefficient, based on the standards within your discipline (e.g., < .65 Poor; .65 ≤ Modest ≤ 80; > .80 Very good)

Reliability Examples: Internal consistency reliability tests were run on a subset of items from the Myers-Briggs Type Indicator Assessment. The extraversion subscale consisted of 10 items (alpha = .77), the introversion subscale consisted of 6 items (alpha = .70), and the agreeableness subscale consisted of 8 items (alpha = .66). Cronbach’s alphas for the 15 items on writing self-efficacy and the 10 items of critical reading were very good at .84 and .88, respectively. Two sets of scores from the 20-item stress inventory were positively correlated (r = .91), demonstrating a strong test-retest reliability.

Writing about validity Establish what kind of validity you have assessed in the instruments you’re using: Common types of validity: Construct validity = extent to which an instrument actually measures what it claims to measure Convergent validity = degree to which a measure is correlated with other theoretically correlated measures Content validity = "the systematic examination of the test content to determine whether it covers a representative sample of the behavior domain to be measured" (Anastasi & Urbina, 1997 p. 114)

Validity Once you’ve figured out what kind of validity to report, here are some other things to keep in mind: Specify the inferential statistics you used to demonstrate validity (e.g., factor analysis, ANOVA, correlation) Provide an interpretation of those results for your current study

Validity Examples: Construct validity on the IQ test was demonstrated through a confirmatory factor analysis from a sample of individuals at an online school. A four-factor model was developed based on fit indices; this model suggested that the instrument measured a verbal, numerical, spatial, and logical intelligence type. Convergent validity on the newly created IQ test was demonstrated through a Pearson product moment correlation test the Wechsler Adult Intelligence Scale (WAIS) IQ instrument (r = .81), illustrating the soundness of the newly created instrument in assessing IQs in the current study sample. Content validity was established by recruiting widely known subject matter experts on intelligence testing (N = 13); these experts evaluated a subset of test items on the proposed IQ test against the test specifications. Overall, their ratings of test items conformed to the dimensions delineated in the test specifications, suggesting a strong content validity in this instrument.

Writing about researcher backgrounds and biases The level of detail you provide about your background as a researcher depends on the general design you’re using in your research. If you’re using a quantitative design, there should be little, if any, disclosure of your background, as the expectation is that your role in the interpretation of your data is kept to a minimum. If you’re using a qualitative design, on the other hand, you should provide enough detail about your background so that your readers can interpret your findings with a full understanding of the lens you used to gather and analyze your data.

Researcher bias Here are some tips for describing your background: Keep this free from emotion. If you have (or had) any preconceived notions of what you might find—or what you hope to find—in your results, disclose those perceptions. There will be readers who will dismiss these disclosures because they admit to “biases.” However, these situate the researcher and inform readers as to where the researcher is speaking.

Researcher bias Example: I feel a connection with adolescents in these groups that stems from my own experience of being labeled an outsider as a high school student. This experience may have contributed to some bias and reactivity in my work. Yet this effect may not have been negative. Indeed, my label as an outsider may have been more of an asset than a liability, drawing out detailed narratives from participants who seemed to view me as a compassionate listener, and conveying a sense of “emotional credibility, vulnerability, and honesty” that has been valued in creative social science work (Ellingson, 2009, p. 154).

The next few slides provide examples of how to write about methods These will be helpful to go back to later and help guide you as you write about your methods.

Introduction In this chapter, I describe the qualitative research paradigm and life history design for this study of adult readers and will discuss the rationale for choosing each in this context. In addition, in this chapter, I describe the methodology for this study, including a description of the participants, how participants were selected, the researcher’s role, and ethical issues. This chapter also includes explanations of the data collection tools, how data was collected and analyzed, and threats to data quality.

Paradigm and Tradition According to Creswell (2006), the qualitative research paradigm should be undertaken based on the following rationales: (a) research questions begin with how and what, (b) the topic requires exploration because of multiple variables and/or a lack of theory, (c) a natural setting is required…Thus, a qualitative research design was chosen because words are more indicative of the experience of learning in reference to the cultural invention of reading than the numerical data of quantitative research… What pertains to your study

Sample According to Champion (2002), purposive sampling is used when there are “clear criteria for selecting the participants for the sample group to be studied” (p. 62). Rather than gathering a random sample of the accessible population from all of the 2-year institutions in Ohio, I employed a purposive sampling of students from the 16 schools that used the COMPASS test for placement... What about the [section] is relevant to your study.

Participants/Population I chose participants for this qualitative life history because they had the shared experience of struggling to learn to read… The participants for this qualitative life story were selected by from a rural central Florida community. A convenience sample of 18 men and women who self-identified as having learned to read as adults was located through (a) referrals from teachers in public school adult education programs, (b) notices sent to community volunteer adult tutoring programs, (c) referrals from the researcher’s professional contacts… Level of detail… repeating the study

Conclusion Over the past several decades, researchers have attempted to determine the conditions under which successful training transfer occurs. In doing so, they found that training transfer is influenced by a number of individual, training design, and environmental factors (Burke & Hutchins, 2007; Blume et al., 2010; Cheng & Hampson, 2008; Cheng & Ho, 2001). Despite the wealth of research that has been conducted, the influence of supervisor support and specific dimensions of support, on training transfer are still unknown (Chiaburu, 2010; Cromwell & Kolb, 2004; Sookhai & Budworth, 2010). The goal of this mixed methods study was to examine the influence of specific dimensions of support (coaching, mentoring, task support, and social support) on training transfer. In order to address this gap, I employed a mixed-methods design in which quantitative data were collected and analyzed. In Chapter 4, the quantitative and qualitative results are presented.

Example of too much information Research Design Phenomenology was not suitable for this study because it would have involved … Ethnography is another form of qualitative research methods. This is a scientific research strategy (Creswell, 2003) often used… Historical research is another type of qualitative research design. It is a systematic collection… Grounded theory is another form of qualitative research design. Grounded theory is a systematic generation of theory… In summary, qualitative research, as an inquiry, is aimed at… There are some factors to consider when a researcher is deciding to adopt a qualitative research methodology. Strauss and Corbin (1990) claimed … Specifically about describing the methods you used.

Example of not enough information Research Design I chose a quantitative research design to demonstrate a correlation between two variables. I did not choose a qualitative study because it would not allow me to do this.

Just right Research Design In this study, I employed mixed methods research. Mixed methods research is “a type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration” (Johnson, Onwuegbuzie, & Turner, 2007, p. 123). Conducting mixed methods research has several advantages compared to conducting quantitative or qualitative research alone, many of which are particularly significant to this study. First, mixed methods research allows for a more complete understanding of complex phenomena. Second, it allows the researcher to compensate for the weaknesses of one method with the strengths of another. For instance, qualitative data can help explain, clarify, and provide meaning to quantitative data. Similarly, quantitative data can limit the influence of confounding variables and increase the generalizability of results. Third, mixed methods research can add to the credibility and validity of findings through the corroboration of qualitative and quantitative findings and by reducing bias related to using one type of methodology (Bryman, 2006; Johnson & Onwuegbuzie, 2004; Kelle, 2006).

Summary Begin writing by finding the appropriate template and rubric or checklist. Go through each required heading and begin writing about the information needed in these areas. Consider your verb tense and voice, and be aware of anthropomorphism. Reflect on your writing as you go Is it clear to the reader what method I used and how I arrived at this choice? Is it clear what actions I took and procedures were involved in my data collection and measurement? Am I adequately citing methods sources?

Writing about the methods is like creating a butter sculpture at the Minnesota State Fair…

The method components should be clear Writing is a process Allow yourself multiple revisions of each section and take advantage of the Writing Center’s resources. The method components should be clear The description of the method should be detailed enough so that others could replicate the study.

Resources: Walden CRQ and Writing Center Center for Research Quality Research resources See Research Planning and Writing See Research Design and Analysis Forms Including templates and rubrics arranged by program: DBA, DNP, EdD, PhD Writing Center Webinars See Scholarly Writing Webinars See Graduate Level Webinars See Capstone Webinars

Writing Center: Writing instruction services To schedule an appointment with a writing instructor to have your proposal chapter/section reviewed: In your myWalden portal, click the Academics tab. On the next page, click Schedule an Appointment. Then click Writing Center and Tutor. Upload your draft. Search for available appointments in the writing instructors’ schedule.

Questions For questions about research analyses consult your chair or email the Center for Research Quality: crq@waldenu.edu Email specific writing or APA questions to the Writing Center: editor@waldenu.edu Photos courtesy CreativeCommons.org

Questions?