2. Surviving Pharmacy Residency Research: Tips and Tricks for Statistical Planning

3. © Fraser Health Authority, 2007 The Fraser Health Authority (“FH”) authorizes the use, reproduction and/or modification of this publication for purposes other than commercial redistribution. In consideration for this authorization, the user agrees that any unmodified reproduction of this publication shall retain all copyright and proprietary notices. If the user modifies the content of this publication, all FH copyright notices shall be removed, however FH shall be acknowledged as the author of the source publication. Reproduction or storage of this publication in any form by any means for the purpose of commercial redistribution is strictly prohibited. This publication is intended to provide general information only, and should not be relied on as providing specific healthcare, legal or other professional advice. The Fraser Health Authority, and every person involved in the creation of this publication, disclaims any warranty, express or implied, as to its accuracy, completeness or currency, and disclaims all liability in respect of any actions, including the results of any actions, taken or not taken in reliance on the information contained herein.

4. FH Health Research Intelligence Unit How can we help?  Grant Facilitator-Writer  Conducting a search for funding opportunities.  Automatic notification of new funding sources and deadlines.  Identifying a research team.  Preparing letters of intent.  Identifying resources required for conducting research.  Formulating the research budget.  Writing the grant application in collaboration with researchers.  Understanding FH and funding agency requirements regarding preparation of specific documents.

5. FH Health Research Intelligence Unit How can we help?  Epidemiologist  Specifying the research goal, objectives and hypothesis.  Identifying measurable outcomes.  Specifying the variables for analysis.  Identifying sources of data.  Developing data collection tools for quantitative or qualitative studies.  Developing the statistical analysis plan.  Understanding how to use statistical software, such as SPSS.

6. Workshop Outline  Research 101- Basic Research Steps  Research Question Refinement  Common Study Designs- Resource  Levels of Data  Power and Sample Size  Statistical Test Selection- Exercise  Data Reporting- Resource  Simple Stats with Excel- Resource

7. Pharmacy Residency Project  1) Develop a research question  2) Conduct thorough literature review  3) Re-define research question or hypothesis  4) Design research methodology/study  5) Create research proposal  6) Apply for funding  7) Apply for ethics approval  8) Collect and analyze data  9) Draw conclusions and relate findings

8. Research Question Refinement  Research question will describe in operational terms, what you think will happen in the study.

9. Good Versus Bad Research Question  Are patients who take drug X more likely to experience episodes of delirium?  Do patients who receive medication X between September 2008 and November 2008 experience more episodes of delirium as compared to patients who received drug Y during the same time period?

10. Classification of Research Studies Research Studies Observational DescriptiveAnalytic Experimental Observational Studies: Descriptive Studies: Focus on describing populations and describing the relationship between variables Analytic Studies: Make inferences about the population based on a random sample. Experimental Studies: Test relationships between exposures and outcomes. Investigator has direct control over study condition and exposure status.

11. Hierarchy of Studies Experimental Studies Analytic Studies Descriptive Studies Type of study is selected according to the purpose of research.

12. Types of Descriptive Studies  Case report: Detailed description of individual participant.  Case series: Extension of case report. Describes characteristics of a group of individuals with same condition/symptoms.  Ecological studies: Can be analytic or descriptive. Group is studied.  E.g. The examination of the association between chlorinated water and cancer among 30 municipalities.  Cross-sectional studies: Assess both exposure and outcome status at one point in time, or during a brief period of time. The individual is the unit of analysis. Can be descriptive or analytic.

13. Types of Analytic Studies  Case-control studies: Subjects selected according to outcome status (present/absent) BEFORE exposure/treatment status is determined. New cases and comparable controls are selected and compared to establish causal relationships. Subjects selected according to outcome status (present/absent) BEFORE exposure/treatment status is determined. New cases and comparable controls are selected and compared to establish causal relationships. E.g. Want to study the relationship between Drug P and lung cancer. Interview participants with lung cancer (cases), and participants without lung cancer (controls) about prior exposure to Drug P. E.g. Want to study the relationship between Drug P and lung cancer. Interview participants with lung cancer (cases), and participants without lung cancer (controls) about prior exposure to Drug P.

14. Case Control Studies  Pros: Quicker to complete as compared to cohort study. Quicker to complete as compared to cohort study. Relatively inexpensive. Relatively inexpensive.  Cons: Prone to selection and recall bias. Prone to selection and recall bias.

15. Types of Analytic Studies  Cohort study: Longitudinal group study  Prospective cohort studies: Subjects classified without study disease or condition, and then followed up into the future to determine if the rate of development of the disease/condition is different in exposed/unexposed groups. E.g. Framingham Study  Retrospective cohort studies: Sample represents group that is assembled by using past data. Subjects classified according to exposure status at the time the cohort existed, and followed up to present, to see if the development of the study disease is significantly different in exposed/unexposed groups.

16. Cohort Studies Pros Pros Yields true incidence rates and relative risks.Yields true incidence rates and relative risks. Cons Cons Expensive, requires large numbers, takes long time to complete, prone to attrition bias.Expensive, requires large numbers, takes long time to complete, prone to attrition bias.

17. Types of Experimental Studies  Randomized Control Trials (RCT): Designed to test efficacy of intervention. Subjects randomly assigned to control and exposure groups, allowing for comparison and the drawing of conclusions.  Community Trial: Assigns interventions/exposure to entire community or group. One group receives intervention, the other group acts as a control group.

18. Quasi-Experimental Studies  Quasi-Experimental: Quasi=Almost Quasi=Almost Lacks random assignment Lacks random assignment Many types: Pretest Posttest Nonequivalent Group Many types: Pretest Posttest Nonequivalent Group Both a control group and an experimental group are compared. But, groups are chosen and assigned out of convenience (rather than randomization).Both a control group and an experimental group are compared. But, groups are chosen and assigned out of convenience (rather than randomization). E.g. Examining two groups of students. One group signs up for a statistics class, one group does not. Would measure all of the students’ grades prior to the start of the class and then again after the program. Those students who participated would be our treatment group; those who did not would be our control group.E.g. Examining two groups of students. One group signs up for a statistics class, one group does not. Would measure all of the students’ grades prior to the start of the class and then again after the program. Those students who participated would be our treatment group; those who did not would be our control group.

19. Quasi-Experimental Studies Important point: How groups are chosen No randomization involved.No randomization involved. Groups often chosen based on convenience.Groups often chosen based on convenience. Not as strong as experimental studies, but often used in health services research.Not as strong as experimental studies, but often used in health services research.

20. Levels of Evidence  Handout- Research Design Hierarchy

21. Probability Sampling Methods: Random  There are several methods to choose from:  Simple random sampling.

22. Probability Sampling Methods: Stratified  Stratified sampling (divide the population into non-overlapping strata and sample from within each stratum independently).  Guarantees representation of all important groups.

23. Probability Sampling Methods: Systematic Probability Sampling Methods: Systematic  Selection of the sample using an interval “k” so that every “k” unit in the frame is selected, is called systematic random sampling.

24. Probability Sampling Methods: Systematic  Steps to achieve a systematic random sample: 1. Number the units in the population from 1 to N. 2. Decide on the n (sample size) that you want or need. k = N/n = the interval size. k = N/n = the interval size. 3. Randomly select an integer between 1 and k. 4. Then take every kth unit. Example: 1. N=200 2. n=40, take N/n, 200/40=5 (interval size). 3. Randomly select a number between 1 and 5 (let’s pick 4). 4. Begin with 4, and take every 5 th unit.

25. Probability Sampling Methods: Cluster  Cluster sampling.  Divide population into clusters and randomly sample clusters.  Measure all units within sampled clusters.  Example: See blue areas on map. Not just geographic areas, Not just geographic areas, could select hospitals, schools etc.

26. Non-Probability Sampling Methods  There are different types of non-probability sampling methods as well: Convenience (not representative of population). Convenience (not representative of population). Purposive (certain group in mind). Purposive (certain group in mind). Expert sampling (seek out specific expertise). Expert sampling (seek out specific expertise). Snowball sampling (ask people to participate, they ask more people). Snowball sampling (ask people to participate, they ask more people). If you select non-probability sampling methods, the conclusions drawn from the study results apply only to that specific population. If you select non-probability sampling methods, the conclusions drawn from the study results apply only to that specific population.

27. Measurement: Levels of Data  The level of data will dictate which statistical test you should use.  Categorical = Data that is classified into categories and cannot be arranged in any particular order (e.g. Apples and pears, gender, eye colour, ethnicity).  Ordinal = Data ordered, but distance between intervals not always equal. (e.g. Low, middle and high income).  Continuous = equal distance between each interval (e.g. 1,2,3., age).

28. Statistics and Statistical Test Selection

29.  Descriptive Statistics: Describes  research findings E.g. Frequencies, averages. E.g. Frequencies, averages.  Inferential Statistics: Makes inferences about the population, based on a random sample. In a random sample, each person/unit has an equal chance of being selected In a random sample, each person/unit has an equal chance of being selected Allows generalizability to population. Allows generalizability to population. Types of Statistics

30. Types of Variables  Variables can be classified as independent or dependent.  An independent variable is the variable you believe will influence your outcome measure.  A dependent variable is the variable that is dependant on or influenced by independent variable(s). The dependent variable can also be the variable you are trying to predict.

31. Selecting the appropriate Statistical test requires several steps: Test selection should be based on: 1) What is your goal? Description? Comparison? Prediction? Quantify association? Prove effectiveness? Prove causality? 2) What kind of data have you collected? What are the levels of data (Nominal, ordinal, continuous)? Was your sample randomly selected? 3) Is your data normally distributed? Should you use a parametric or non- parametric test? 4) What are the assumptions of the statistical test you would like to use? Does the data meet these assumptions? Statistical Test Selection

32. Parametric Tests  Parametric tests assume that the variable in question is from a normal distribution.  Non-parametric tests do not require the assumption of normality.  Most non-parametric tests do not require an interval level of measurement; can be used with nominal/ordinal level data.

33. Assumptions  There are various assumptions for each test.  Before you select a test, be sure to check the assumptions of each test.  You will need to contact a consultant, or review statistical/research methods resources to find this information.  Some examples of common assumptions are: The dependent variable will need to be measured on a certain level (i.e. Interval level). The dependent variable will need to be measured on a certain level (i.e. Interval level). The independent variable(s) will need to be measured on a certain level (i.e. Ordinal level). The independent variable(s) will need to be measured on a certain level (i.e. Ordinal level). The population is normally distributed (not skewed). The population is normally distributed (not skewed).  If your data do not meet the assumptions for a specific test, you may be able to use a non-parametric test instead.

34. Type of Data Goal Measurement Normal Population Ordinal, or Non- Normal Population Binomial -Two Possible Outcomes Survival Time Describe one group Mean, SD Median, interquartil e range Proportion Kaplan Meier survival curve Compare one group to a hypothetic al value One-sample t test Wilcoxon test Chi-square or Binomial test ** Compare two unpaired groups Unpaired t test Mann-Whitney test Fisher's test (chi- square for large samples) Log-rank test or Mantel-Haenszel* Compare two paired groups Paired t testWilcoxon test McNemar's test Conditional proportional hazards regression* Compare three or more unmatched groups One-way ANOVA Kruskal-Wallis test Chi-square test Cox proportional hazard regression** Compare three or more matched groups Repeated- measures ANOVA Friedman testCochrane Q** Conditional proportional hazards regression** Quantify association between two variables Pearson correlation Spearman correlation Contingency coefficient s** Predict value from another measured variable Simple linear regression or Nonlinear regression Nonparametric regression** Simple logistic regression * Cox proportional hazard regression* Predict value from several measured or binomial variables Multiple linear regression* or Multiple nonlinear regression** Multiple logistic regression * Cox proportional hazard regression*

35. Statistical Test Selection Group Exercise  Using your tables, select the appropriate statistical tests for 10 research scenarios.  Handout- Test Selection Exercise

36. During the group exercise…  Steps to choose the appropriate statistical method for the data analysis: 1. Identify whether the research problem raises the question of describe, relate (association), or compare (difference). 2. Identify the levels of measurement in the research question (Nominal/Categorical, Ordinal/Rank, Continuous/Evenly spaced). 3. Identify the number of variables, or samples being described, related, or compared. 4. Identify whether comparison samples are related (analyze same group before and after) or independent (not at all related, looking at different groups). 5. Choose the appropriate statistical tool for the data and situation using the decision tree in the handout.

37. What is the question: Compare How many samples: 2 Related or independent: Independent What is the level of measurement: Continuous How many dependent variables: 1 Test: T-test  1. A pilot experiment designed to test the effectiveness of a new approach to electrode placement for Electro Shock Therapy (ECT) has been conducted over a one year time period in the Fraser Health Authority. Patients from two different mood disorder clinics participated in this study. Patients from Clinic X received ECT therapy according to current practice guidelines. Patients from Clinic Y received a new exploratory ECT treatment. Patients in each clinic were matched for age, gender, and type of disorder. A random sample of 30 matched pairs of patients were selected for inclusion in the study. At end of one year, patients were administered a memory test yielding a total score out of 100. Dr. Vasdil would like to know what statistical procedure needs to be selected to test for differences among groups of patients on the memory test.

39. Sample Size  There are several rules of thumb for determining sample size.  1) It’s a good idea to have a minimum of 30 cases (as a total group, or if comparing groups, 30 for each group).  If you have less you can use a non-parametric test, but it is still better to have close to 30 cases.  2) If using regression, it is best to have between 10-50 cases per independent variable.  3) If you are validating a survey, it is never good to have more questions than cases.  4) If the total population that you are examining is less than 30. Use all of them.  5) For pilot studies the recommendation is a sample size of 12 per group  6) For surveys, a sample size of 400 per group can do just about anything.  7) For surveys, a 30% response rate is the bare minimum.  Note: For a precise sample size estimate you will need to conduct a power analysis.

40. Statistical Power  Power is the capability of a statistical test to correctly detect a significant effect if it exists.  Assumes value between 0 and 1 (%)  Power= 1-B (B= probability of a Type II error).  Type II error – the error of not rejecting a false research finding.  Type I error- the error of rejecting a correct research finding.

41. Types of Power  A Priori- Conducted before study commences (at proposal stage).  Post Hoc- After study has been completed.  Easy way to increase power? Increase sample size Increase sample size Increase Effect size Increase Effect size

42. Components Involved in Power Calculation  Sample Size- Number of cases.  Effect Size –Magnitude of the trend and variation.  Alpha Level- Odds of concluding that the presence of an effect is due to chance alone (.05 or.01). Also known as Type I Error, or the error of rejecting a correct research finding Also known as Type I Error, or the error of rejecting a correct research finding  Power level- 80-90% common  One or two-tailed test- two tailed is common.

43. Components Involved in Power Calculation  Sample Size- What we want to find out.  Effect Size –Magnitude of the trend…but what if you don’t know? Look to pilot data or literature. Look to pilot data or literature. Keep in mind, the smaller the effect size, the larger the sample size required. Keep in mind, the smaller the effect size, the larger the sample size required.  Alpha Level-.05  Power level- 80-90%

44. Important Consultation Information  What is your research question?  Components of power calculation  Levels of data (nominal, ordinal, continuous)  Sampling plan

45. Data Organization: Codebook  What is a codebook?  A codebook is a log of your variables (and levels of data) and how you will code them.  A codebook will help everyone understand the coding schemes to ensure that they are on the same page!

46. Data Processing and Analyses: Codebook Example Variable Name NameVariable Label LabelValuesCodingMissingVariable Type Type ageage1,2,3,4,5 1=10-20 years 2=21-30 years 3=31-40 years 4=41-50 years 5=51+ years 97=Incorrect response 98=No response 99=Not Applicable Ordinal sexsex1,2 1=male, 2=female 97=Incorrect response 98=No response 99=Not Applicable Nominal happiness happiness at work work1,2,3 1=not happy 2=somewhat happy 3=very happy 97=Incorrect response 98=No response 99=Not Applicable Ordinal

47. Spreadsheet Example ID# Age Sex Happiness 1112 2222 3312 45722 54523 66623 7223 88823

48. Data Analysis with Excel  Most simple analyses can be done using Excel, including correlation, regression and even random number generation.  Install the data analysis pack. Go to tools, add-ins, and add the ‘analysis tool pack’. Go to tools, add-ins, and add the ‘analysis tool pack’.  Create worksheet and codebook.  Choose statistical test. Follow commands in help menu. Follow commands in help menu.

49. http://home.ubalt.edu/ntsbarsh/excel/excel.htm http://home.ubalt.edu/ntsbarsh/excel/excel.htm http://home.ubalt.edu/ntsbarsh/excel/excel.htm Data Analysis with Excel

50. Data Reporting and Presentation of Data  Graphical summaries are a great way to present your data  Excel is great for creating tables and graphs  The type of data you have will reflect the type of graphical summary you should use.

51. Data Reporting and Presentation of Descriptive Data  Categorical data: Frequency Tables and Bar Charts.  Example: Fruit CountPercent Valid Percent Pineapples420%21% Apples525%26% Oranges1050%53% Unknown15%_______ Total20100%100%

52. Data Reporting and Presentation of Descriptive Data

53.  Continuous Data: Tables and Histograms AgeCountPercent20-30420% 31-40525% 41-501050% 51-6015% Total20100%

54. Data Reporting and Presentation of Descriptive Data 20304050

55. What is the difference between a Histogram and a Bar Chart?  Histogram: For continuous data where data are divided into contiguous class intervals (or in other words, connected through unbroken sequence).  Bar Chart: For categorical data where categories are not contiguous.

56. Measures of Central Tendency  Reporting averages Categorical data= Mode Categorical data= Mode Ordinal data= Median Ordinal data= Median Continuous data= Mean Continuous data= Mean  If there are outliers (or extreme values), report the median instead of the mean.

57. Reporting Inferential Stats  Handout Resource- APA Guidelines  http://www.ilstu.edu/~jhkahn/apastats.html http://www.ilstu.edu/~jhkahn/apastats.html

58. Reporting Inferential Stats  It’s important to include means, standard deviations and sample size in your results section.  Example: Correlation  Variable X was strongly correlated with Variable Y, r=.59, p<.01.

59. Important to Keep your Audience in Mind Residency Project Publication Departmental Report

60. Aaron: TCPS certification for residents reminder…

61. Questions?