Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort 4 th Meeting, Paris, July 20, 2011 Principal Investigators: Giovanni B Frisoni & Clifford R Jack Project Coordinator: Marina Boccardi

Housekeeping Information Webinar attendees will be muted to limit surround noise Please write your questions in the “Questions” window. Your question will be forwarded to the speakers.

EADC F Barkhof/P ScheltensAmsterdam B Dubois/S LehericiParis N Fox/ J BarnesLondon GB Frisoni Brescia H Hampel/J PantelFrankfurt C HockZurich A SimmonsLondon H SoininenKuopio S TeipelRostock LO WahlundStockholm THE WORKGROUP ADNI CR Jack Jr Rochester, MN M AlbertBaltimore, MD G Barzokis UCLA, CA D Bennet Chicago, IL J Csernansky NorthW U, WA C DeCarli UC Davis, CA M De Leon New York, NY L DeToledo-Morrell Chicago, IL J Kaye Portland, OR R Killiany Boston USM, MA PM Thompson LoNI, UCLA, CA M Weiner/S Mueller UCSF/VAMC OTHERS R Camicioli/NV Malykhin Edmonton, AL J O’BrienNewcastle, UK J PruessnerMontreal, QC C Watson Detroit, MI

THE WORKGROUP Population-based studies P Sachdev/JJ Maller PATH Through Life Study T Den Heijer Rotterdam Scan Study L Launer Honolulu Asia Aging Study W Jagust SALSA Study Mirjam I. GeerlingsSMART MEDEA Study Advisory board EADC P.I.s B WinbladStockholm Lutz Froelich, Mannheim ADNI P.I. M WeinerUCSF, US Statistical Board S DuchesneQuébec City L CollinsMontreal P PasqualettiRome Clinical Advisor PJ VisserMaastricht Dissemination/Education G WaldemarCopenhagen

Funded by Alzheimer’s Association grant n Funded in part thanks to an unrestricted grant by Lilly and Wyeth/Pfizer Bartzokis Research support – Janssen; Consultant - Lilly, Pfizer, Novartis; Csernansky Consultant Eli Lilly; Duchesne Research support - Agfa HealthCare Inc.; funding as a partner in the project; Frisoni Consultant Wyeth; Jack Contract – Pfizer; consulting – Élan; Lehericy Consultant - EISAI, Janssen-Cilag DISCLOSURES

Preliminary Phase - Survey of literature - Certified extraction of landmarks (Boccardi et al., J Alzheimers Dis, in press)

Preliminary Phase

- Survey of literature - Certified extraction of landmarks (Boccardi et al., J Alzheimers Dis, in press) - Operationalization of differences (Segmentation Units, SUs) - Quantification of relevant features of SUs (reliability, impact on hippo volume, informative value for AD-related atrophy) - Quantitative data used for Delphi Questionnaires

Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) 5 expert tracers 20 naive tracers The best 5 naive tracers Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) GOLD STANDARD Local Protocol : 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) 1 tracer Qualification global and local 95% confidence intervals RM-ANOVA: test of rater and rater by center terms RM-ANOVA: test of main effects side, trace- retrace, atrophy, time, scanner, rater Local Protocol : Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Harmonized Protocol : Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) RM-ANOVA: test of protocol main effect VARIABILITY EVALUATION VALIDATION vs PATHOLOGY Training (tracing 20 hippos on 1.5T ADNI scans with each SU) Delphi panel → harmonized prot

Project Steps Completed so far - 2 Delphi rounds - Tracing with local protocol (7/20) - Master tracers practice (4/5) - Pilot platform for tracers qualification (will be reported in the interim report for September)

Delphi Panel Murphy et al., Health Technology Assessment, 1998:2(3)

Delphi Panelists PI Delphi Panelist Barkhoff/ScheltensWouter HennemanGeorge Bartzokis Richard CamicioliRichard Camicioli / Nikolai Malikhin John CserrnanskyLei WangCharles DeCarliLeyla DeToledo-Morrell Nick FoxJosephine Barnes Mirjam GeerlingsLotte Gerritsen Clifford JackClifford Jack / Greg PreboskeRonald Killiany John O’BrienMichael FirbankJens Pruessner Hilkka SoininenHilkka Soininen / Maija Pihlajamäki Paul ThompsonLiana ApostolovaCraig Watson Christoph HockHenrike Wolf

Delphi method First round: Questions: Choose based on experience and helped by the quantitative data; Motivate answers. Space for free comments/criticism/suggestions. Second round: Feedback about answers and reasons; Level of agreement about reasons converging towards most frequent answers. We answer previous comments / criticism / suggestions. Space for new free comments/criticism/suggestions.

Delphi Panel – Landmarks Statistics by Patrizio Pasqualetti, Rome

Delphi Panel – I Round

Delphi I Round – Alveus/Fimbria Inclusion P= %69% P = exact p at Binomial Test (2x2) 0 = Exclude Alveus/Fimbria 1 = Include Alveus/Fimbria Frequency

Delphi II Round – Alveus/Fimbria Inclusion 1 = Minimum agreement TO 9 = Maximum agreement P 1 < P 2 =0.021 Median = 9 19% 81% P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test ( 5) (2x2) Frequency

Delphi I Round – Which criterion for subiculum

Delphi I Round – Subiculum P= % 6% 38% 56% 0 = No subiculum 1 = Oblique line criterion 2 = Horizontal line criterion 3 = Morphology criterion P at Fisher’s Exact Test (2x4) Frequency

Delphi II Round – Subiculum Morphology 1 = Minimum agreement TO 9 = Maximum agreement * Explicitly expresses no strong objection for morphology P 1 = P 2 =0.057 Median = 7 19% (12%) 69% * P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test ( 5) (2x2) Frequency

Delphi Panel I Round

Delphi I Round – Tail P= % 25% 69% 1 = No tail 2 = Crura criterion 3 = Crura+End Tail criterion P at Fisher’s Exact Test (2x3) Frequency

Delphi II Round – Tail End 1 = Minimum agreement TO 9 = Maximum agreement P 1 = P 2 =0.035 Median = 8 19% (6%) 75% P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test ( 5) (2x2) Frequency

Delphi Panel – Preliminary Results P=0.021 P=0.057 P=0.035 Covers 100% of hippocampus proper Captures 100% of AD-related atrophy P for agreement among panelists) Intra-rater (BS-BS): Inter-rater (BS-BS): Preliminary ICC (BS-Mayo-LONI): 0.95

Delphi – Preliminary Results This model includes some tissue which is not hippocampus proper: - A/F is white matter - Tail end may include some vestigial tissue - The medial boundary of the body may include some entorhinal cortex

Delphi II Round – Harmonized Hippo P 1 < P 2 =0.001 Median = 8 6% 6% 88% 1 = Minimum agreement TO 9 = Maximum agreement § * Panelist who voted for different criterion for every landmark, all reasons reported for other panelists § Panelist who has problem to agree because some boundaries not discussed (we only questioned landmark heterogeneities in the available literature) * P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test ( 5) (2x2) Frequency

Delphi Panel - Segmentation Modalities

Delphi I round – Disambituating Hippo/Amy in 3D P= %88% 0 = (sometimes) not possible to disambiguate with 3D visualization * 1 = Possible to disambiguate with 3D visualization * Suggested editing in 3D P = exact p at Binomial Test (2x2) * Frequency

Delphi II Round – Disambiguating Hippo/Amy in 3D P 1 < P 2 < Median = 8 0% 0% 100% 1 = Minimum agreement TO 9 = Maximum agreement P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test ( 5) (2x2) Frequency

Delphi I round – Disambiguating Vestigial in 3D P=1 50% 0 = not possible (or opportune) 1 = possible (or opportune) * Proposed again into two questions in the II round P = exact p at Binomial Test (2x2) Frequency

Delphi II Round – Possibility to exclude vestigial tissue in 3D 0 = No 1 = Yes P = exact p at Binomial Test (2x2) P= % 56% Frequency

Delphi II Round – Opportunity to exclude vestigial tissue (with 3D) 1 = Minimum agreement TO 9 = Maximum agreement P 1 = P 2 = Median = % 12% 69% P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test ( 5) (2x2) Frequency

Delphi I Round – Internal CSF P 2 = % 44% 25% 19% 1 = Include CSF in segmentation 2 = Exclude when connected * 3 = Exclude always 4 = Other ** * Criterion to be sure that it is CSF ** Should use other criterion to be sure it is CSF, then exclude P 1 for exclusion (somehow) P 1 =0.004 P 1 = exact p at Binomial Test (2x2: 1 vs 2-4) P 2 = Fisher’s exact test (2x4) Frequency

P 1 = P 2 = Median = 7 31% 6% 63% 1 = Minimum agreement TO 9 = Maximum agreement Delphi II Round – Exclude internal csf when connected P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test (2x2) Frequency

Delphi I-II Round – Not visible structures Trace only if you see, but train to look for tissue there?

Delphi I Round – Plane of tracing 1 = Long axis of the hippocampus 2 = AC-PC 3 = Other * * Both are fine P= % 31% 19% P at Fisher’s Exact Test (2x3) Frequency

Delphi II Round – Hippo long axis 1 = Minimum agreement TO 9 = Maximum agreement * * both AC-PC or long axis hippo are ok P 1 = P 2 = Median = 8 6% 31% 62% P 1 = Fisher’s exact test (2x9) P 2 = exact p at Binomial Test (2x2) Frequency

Discussion - Landmarks Alveus/Fimbria: significant agreement for inclusion (not Hippo proper, but > reliability, and some increase in sensitivity). Subiculum: non significant agreement for morphology; Morphology > Horizontal, but further round needed; Possible inclusion of non hippocampal tissue would apply to both criteria. Tail End: (almost significant) agreement for inclusion of the whole tail; Harmonized Hippo: significant agreement for inclusion of all SUs and using the morphology criterion for subiculum. OK 3° round

Disambiguation hippo/amy in 3D: significant agreement; Suggestion of “editing” in 3D will be considered in further steps. Disambiguation of vestigial tissue: no agreement on the possibility or opportunity to exclude it from the whole hippocampal structure. CSF exclusion: significant agreement on exclusion. no agreement on using the “connection” criterion to be sure. Suggestions will be used to seek consensus on more articulated ways to recognise for sure the CSF and thus exclude it. Discussion – Segmentation Modalities OK OK + 3° round

Segmentation of not visible structures: agreement on inclusion of tissue only if visible Plane of tracing: significant agreement on long axis of the hippocampus. Further rounds will better specify (left, right, mean?) Discussion – Segmentation Modalities OK OK + 3° round

- Landmarks not discussed: because no disagreement in the literature. -Partial volumes: interesting issue, proposed to panelists, but only 2 answered. Will formulate an item based on these 2 in next trial. Discussion – Other Issues

- Multitracer (free; 3D simultaneous visualization; sub-voxel segmentation and volume computation; tracing of different labels on each slice) - Possibility of editing in 3D may be helpful -Please send your suggestions about a free software fulfilling all (or more!) of these needs - Zoom requirements are specific to this project due to common image processing, but do not necessarily apply to the future requirements for the harmonized protocol Discussion – Software

Acknowledgements Mike & Barbara Urbut, Stuart & Amy Savitz, Harriet K. Burnstein, Chicago, IL Maria Carrillo, Meredith McNeil Alzheimer’s Association, Chicago IL Martina Bocchetta, Daniele Tolomeo, Gabriele Corbetta (Brescia, Italy) Patrizio Pasqualetti (Rome, Italy) All partners!