Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomized Studies in Meta- Analysis G. Wells, B. Shea, D. OConnell, J. Robertson, J.

Similar presentations


Presentation on theme: "The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomized Studies in Meta- Analysis G. Wells, B. Shea, D. OConnell, J. Robertson, J."— Presentation transcript:

1 The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomized Studies in Meta- Analysis G. Wells, B. Shea, D. OConnell, J. Robertson, J. Peterson, V. Welch, M. Losos, P. Tugwell

2 Development Applications Current Developments

3 Development: Item Selection Newcastle quality assessment form Ottawa comprehensive list Panel review Critical review by experts

4 Development: Grouping Items Cohort studies Selection of cohorts Comparability of cohorts Assessment of outcome Case-Control studies Selection of case and controls Comparability of cases and controls Ascertainment of exposure

5 Development: Identifying Items Identify high quality choices with a star A maximum of one star for each h item within the Selection and Exposure/Outcome categories; maximum of two stars for Comparability

6

7

8 Newcastle-Ottawa Quality Assessment Scale: Case-Control Studies Selection (4) Comparability (1) Exposure (3) –A study can be awarded a maximum of one star for each numbered item within the Selection and Exposure categories. A maximum of two stars can be given for Comparability

9 1. Is the case definition adequate? a) yes, with independent validation b) yes, eg record linkage or based on self reports c) no description 2. Representativeness of the cases a) consecutive or obviously representative series of cases b) potential for selection biases or not stated 3. Selection of Controls a) community controls b) hospital controls c) no description 4. Definition of Controls a) no history of disease (endpoint) b) no description of source Selection >1 person/record/time/process to extract information, or reference to primary record source such as x-rays or medical/hospital records e.g. ICD codes in database or self-report with no reference to primary record or no description

10 Comparability 1. Comparability of cases and controls on the basis of the design or analysis a) study controls for ___________ (select the most important factor) b) study controls for any additional factor (This criteria could be modified to indicate specific control for a second important factor.)

11 Exposure 1. Ascertainment of exposure a) secure record (eg surgical records) b) structured interview where blind to case/control status c) interview not blinded to case/control status d) written self report or medical record only e) no description 2. Same method of ascertainment for cases and controls a) yes b) no 3. Non-Response Rate a) same rate for both groups b) non respondents described c) rate different and no designation

12 Newcastle-Ottawa Quality Assessment Scale: Cohort Studies Selection (4) Comparability (1) Outcome (3) –A study can be awarded a maximum of one star for each numbered item within the Selection and outcome categories. A maximum of two stars can be given for Comparability

13 Selection 1. Representativeness of the exposed cohort a) truly representative of the average ___________ (describe) in the community b) somewhat representative of the average ___________ in the community c) selected group of users eg nurses, volunteers d) no description of the derivation of the cohort 2. Selection of the non exposed cohort a) drawn from the same community as the exposed cohort b) drawn from a different source c) no description of the derivation of the non exposed cohort 3. Ascertainment of exposure to implants a) secure record (eg surgical records) b) structured interview c) written self report d) no description 4. Demonstration that outcome of interest was not present at start of study a) yes b) no In the case of mortality studies, outcome of interest is still the presence of a disease/ incident, rather than death; that is a statement of no history of disease or incident earns a star

14 Comparability 1. Comparability of cohorts on the basis of the design or analysis a) study controls for ___________ (select the most important factor) b) study controls for any additional factor (This criteria could be modified to indicate specific control for a second important factor.)

15 Outcome 1. Assessment of outcome a) independent blind assessment b) record linkage c) self report d) no description 2. Was follow up long enough for outcomes to occur a) yes (select an adequate follow up period for outcome of interest) b) no 3. Adequacy of follow up of cohorts a) complete follow up - all subjects accounted for b) subjects lost to follow up unlikely to introduce bias - small number lost - > ___ % (select an adequate %) follow up, or description of those lost) c) follow up rate < ___% (select an adequate %) and no description of those lost d) no statement

16 Applications: Assess quality of nonrandomized studies Incorporate assessments in interpretation of meta-analytic results Design, content and ease of use

17 Long Term Hormone Replacement Therapy and Coronary Heart Disease Events

18 Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured report Steps of a Cochrane Systematic Review

19 Objective Is there a relationship between hormone replacement therapy and the incidence of coronary heart disease in postmenopausal women

20 Inclusion Criteria Types of studies –case-control, cohort or cross-sectional studies Population –postmenopausal women Intervention –women exposed to hormone replacement therapy (estrogen or estrogen + progesterone) –ever, current, past Outcomes –coronary heart disease (events) –fatal, non-fatal, both

21 Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured report Steps of a Cochrane Systematic Review

22 Search Strategy Electronic Search of: –MEDLINE (1966 to May 2000) –Current Contents (to May 2000) Other Data Sources: –review of references cited in retrieved articles

23 Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured report Steps of a Cochrane Systematic Review

24 Data Extraction 2 independent reviewers selected trials 2 independent reviewers extracted data using pre-determined forms –study design –population characteristics –exposure to implants –outcomes measures –results differences resolved by consensus

25 Results 16 case-control or cross-sectional 14 cohort

26 Quantification of Effects Exposure (ever, current, past) Outcome (fatal, non-fatal, both) Effect estimates (EE) Relative Risk (RR) Odds Ratio (OR) Adjusted effect estimates Effects vs population, follow-up periods, etc. (homogeneity)

27 Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured report Steps of a Cochrane Systematic Review

28 Avila / 90 Cauley / 97 Grodstein / 96 Henderson / 91 Lafferty / 94 Wilson / 85 Wolf / 96 Lauritzen / 83 Ettinger / 96 Petitti / 87 Sourander / 98 Bush / 87 Criqui / 98 Folsom / 95 Cohort Star Template Selection Comparability Outcome

29 Adam / 81 Beard / 89 Croft / 89 Grodstein / 97 Heckbert / 97 LaVecchia / 87 Mann / 94 Pfeffer / 78 Rosenberg / 76 Rosenberg / 80 Rosenberg / 93 Ross / 81 Sidney / 97 Szklo / 84 Talbott / 77 Thompson / 89 Case-Control Star Template Selection Comparability Exposure

30 Adjusted Effect Estimates for Coronary Heart Disease (All Events) (HRT: Estrogen Current Use) Case-Control Studies Selection Comparability Exposure

31 Adjusted Effect Estimates for Coronary Heart Disease (All Events) (HRT: Estrogen Past Use) Case-Control Studies Selection Comparability Exposure

32 Adjusted Effect Estimates for Coronary Heart Disease (All Events) (HRT: Estrogen Ever Use) Case-Control Studies Selection Comparability Exposure

33 Adjusted Effect Estimates for Coronary Heart Disease (All Events) (HRT: Estrogen + Progestin Ever Use) Case-Control Studies Selection Comparability Exposure

34 Adjusted Effect Estimates for Coronary Heart Disease (All Events) (HRT: Estrogen Current Use) Cohort Studies Selection Comparability Outcome

35 Adjusted Effect Estimates for Coronary Heart Disease (All Events) (HRT: Estrogen Ever Use) Cohort Studies Selection Comparability Outcome

36 Current Development: Validity Face/content validity Criterion validity compare to more comprehensive scales compare to expert judgement Construct validity external criteria –convergent validity –divergent validity internal structure –factorial validity

37 Current Development: Reliability Inter-rater reliability Intra-rater reliability

38 Future Development: Scoring Identify threshold score distinguishing between good and poor quality studies

39 The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomized Studies in Meta- Analysis www.lri.ca NOS Quality Assessment Scales: Case-control studies Cohort studies Manual for NOS Scales


Download ppt "The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomized Studies in Meta- Analysis G. Wells, B. Shea, D. OConnell, J. Robertson, J."

Similar presentations


Ads by Google