2 What is operational research? (OR) Definitions found on the internet: –"Mathematical common sense" –"Systematic study, by observation and experiment, of the working of a system, e.g. health services, with a view to improvement" –"Using scientific methods to attack a complex problem or system"
3 In the beginning there was... a question "Why in the world is it that 30% of our patients on treatment default?" NTP manager in the morning
4 Description of defaulters in Russia 1 Profession: unemployed: 26%, labourers 21%, students of vocational schools 19%, disabled 7% Education: incomplete secondary education: 70% Residence: homeless 5%, >5km away from treatment site 26% Behavioural risk factors: alcoholism 44% 1 Data are from W. Jakubowiak, Russia
5 Are these variables risk factors for default? – use of patient cohort for cohort study
6 Social support system Examples from different oblasts: –Food incentives –Hygienic kits –Free transportation –Psychological support –....
8 More examples "Defaulting from anti-tuberculous treatment in a teaching hospital in Rio de Janeiro, Brazil" (IJTLD 2004) "A concurrent comparison of home and sanatorium treatment of PTB in South India" (BWHO 1959) " 'Lost' smear positive PTB cases: where are they and why did we lose them?" (IJTLD 2005)
9 Determinants of a study Problem or question Data available Funding and staff available Political or hierarchical support Type of study
10 Which scientific methods can we use? - Type of studies Descriptive studies –Analysis of surveillance data –Ecological study (correlational) –Cross-sectional survey Analytical studies –Observational (case-control study, cohort study) –Experimental Other –E.g. capture-recapture study
11 Smear-positive diagnosis by province, Syria 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 AArDHIKLMOSQRRDSSWTZ ss+/all pulmonary Example: Surveillance data reveal large provincial differences of ss+ TB/all PTB
12 No. of slides/patient is correlated with proportion of ss+/PTB
13 Ecological comparison (correlational) Correlation of aggregated or group data Association on the individual level is unknown and may be different Many relationships on global level are strictly speaking of ecological nature
14 Example of an "ecological" comparison: The prevalence of HIV in TB patients (y-axis) against the prevalence of HIV in adults (x-axis).
15 Cross-sectional survey Collection of representative data Based on sampling size calculations, sampling frame and sampling scheme –Simple random sample –Systematic sampling –Cluster sample (design effect!)
16 Surveys are frequently used in TB epidemiology Sampling universe is the population: –Prevalence surveys –Tuberculin skin test surveys Sampling universe is "all TB patients" –Proportion of diagnosed new TB patients with HIV test Sampling universe is the number of culture positive TB patients –Drug resistance surveys
17 Analytical studies Are used to identify risk factors or other forms of "exposure" and their association with an outcome, e.g. death, default, etc. Make use of a comparison group Hypotheses are tested Null hypothesis: "There is no association of exposure and outcome" or: "Exposure and outcome are independent" We then calculate the probability that this is true based on the data
18 Case control study Starts with a group of cases, i.e. with a certain outcome, that is consistent with a case definition The case definition must be specific in regards to time, place and person E.g. "a person with smear positive TB diagnosed in Geneva city in 2004" Then select a group of persons without the outcome from the same population, here for example the general population From the case definition it follows: "a person without TB living in Geneva in 2004"
19 Case control study: ascertainment of exposure status After identification of cases and controls the exposure status preceding the outcome is investigated E.g.: income (high versus low) Thus, the directionality is usually retrospective
20 Selection of controls Imagine the cohort from which the cases would have arisen Or: Would the control have been a case if he/she had had the outcome in question? Example: cases of rare kidney disease in the Mayo clinic
22 2 x 2 table (CCS (1)) 50/1000 ss+ TB cases (5%) were poor, but only 5 of 2000 (0.25%) among the non-TB persons Ss+ TB patients were 20 times more likely than the general population to be poor, however...
23 2 x 2 table (CCS (2)) The chances of ss+TB patients to be poor is expressed as the odds = probability of poverty / prob of rich = 50/1000 / 950/1000 = 0.053 The odds of non TB persons for poverty is therefore: 5/2000 / 1995/2000 = 0.00251 The ratio of the two odds (the odds ratio (OR)) is: 0.053/0.00251 = 21
24 Use of case control studies When type of outcome is rare We can examine >1 exposure Usually relatively quick and inexpensive Disadvantages: –Not useful for rare exposures –Because exposure is in the past: watch out for recall bias –Selection of cases and controls often not straightforward (selection bias)
25 Cohort study Starts with a group of people or a population that can be divided in two groups based on a defined exposure which some have and some don't The groups are then followed-up and an outcome is counted A case definition is still important The directionality is usually forward, but can also be backwards (retrospective cohort study)
26 2 x 2 table (cohort study) We follow 100 low income TB patients and 200 high income TB patients up for adverse outcomes It turns out that 20 of 100 (20%) poor have a bad outcome versus 10 of 200 (5%) of the rich. Thus, the poor are 4 times more likely to have an adverse treatment outcome. Measure of association is the risk ratio (RR) = 0.2/0.05 = 4
27 Use of cohort studies When exposure is rare We can examine >1 outcome The outcome measure for the strata is an incidence rate or (cumulative) risk and the overall point estimate the rate ratio or risk ratio (RR) Disadvantages: –Not suitable for rare outcomes –Not ideal for outcomes in the far future (unless you have much time or lots of scientific altruism) –Watch out for loss to follow-up (they may represent a certain category of patients)
28 The TB quarterly "cohort" Pro- or retrospective cohort study (Nested) case-control study Alhocol addiction No addiction Default?/Cure? Cases Controls Default Cured Alhocol addiction No addiction Information may be available from start of treatment Pro- Retro-
29 2 x 2 table exposed (unprotected) not exposed (protected) exposed (unprotected) not exposed (protected) HEALTHY DISEASED ill Not ill exposed not exposed
30 Cohort study ill healthy exposed not exposed time
31 Case control study ill healthy exposed not exposed time
32 Analytical study: experimental / intervention study Prospective Use of a cohort Exposure is usually an intervention, a drug or vaccine Patients are ideally randomized which guarantees minimisation of bias Example: IPT intervention study in South African gold miners; recruitment in random sequence; comparison before / after IPT phase
33 Steps for a OR protocol (1) Starts with a problem or question: e.g. "Why is there no decline in urban TB in Japan?" Gathering of information: –Analyse exhaustively routinely collected (surveillance) data and disaggregate also by province etc –Talk with stakeholders –Investigation of the literature –Contact other countries Develop a hypothesis Depending on money and staff available: generate a protocol; but this can also be used to generate money and staff
34 Steps for a OR protocol (2) Writing of the protocol: –You can structure it similar to a scientific paper –Introduction/rationale –Objective –Methods (study type, sample size, case definitions used, inclusion/exclusion criteria, training, data collection, data entry (double entry?, data validation), quality control, lab methods, method of analysis) –Ethical considerations –Results: shell tables, expected figures –Timeline –Budget –Appendices (questionnaire, maps, consent form...) –Good idea to do a pilot: feasibility, cost, first crude data verify sample size assumptions