Agenda: - Axes check - Validation vs Pathology - Paper Publications - Congress presentations & Meetings VII PMT meeting – Feb 29, 2012.

Slides:



Advertisements
Similar presentations
Evaluating and Institutionalizing
Advertisements

From Learning Objectives to Outcomes Assessment During Simulation Paul E. Phrampus Director Winter Institute for Simulation, Education and Research (WISER)
Data Collection Strategies An Overview. How Do We Assess ?
Agenda: - Certification Platform: performance for training - I round - II round -Feedback from naives - Beta testers proposals - Congress presentations.
J Garza Consulting and Associates 1 Pilot Car Escort Certification Model For State Implementation.
Agenda: - Update master tracers - Update validation vs pathology - Hippocampal 3D Object for AR Visualization - Papers - Next congress presentations &
Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort 5 th Meeting, New Orleans, April 24, 2012 Update as of April 2012.
Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort 4 th Meeting, Paris, July 20, 2011 Principal Investigators: Giovanni.
Agenda: - Validation vs Pathology - Congress presentations & Meetings - Paper publication VI PMT meeting – Feb 1, 2011.
Answers to previous TC issues: - Demographic features of the selected subjects (age, sex, disease, ApoE) - Check for SIEMENS-Allegra scans at 3T for the.
Navigation-related structural change in the hippocampi of taxi drivers
AAN –Toronto, April 11, 2010 Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort INTRODUCTION.
Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort 3 rd Meeting, Honolulu, April 13, 2011 Principal Investigators:
Agenda: - Hippocampal Label Expansion project - Certification Platform: performance/criteria - Questionnaire - Congress presentations - Papers XV PMT meeting.
Agenda: - Update on master tracers’ segmentation check - Update validation vs pathology - PMT application submission of “Study on the validation of VSRAD”
National Alliance for Medical Image Computing Core DBP Dartmouth Data NA-MIC National Alliance for Medical Image Computing
Agenda: - New benchmark ICC and Dice’s - Training set for Naïve - Beta testers proposals - Congress presentations XII PMT meeting – September 26, 2012.
8. Evidence-based management Step 3: Critical appraisal of studies
Agenda: - Validation Harmonized Protocol: update - Validation vs pathology - Hippocampal Label Expansion project: update - News - Congress presentations.
Agenda: - Certification Platform: Training - I round - Beta testers proposals - Congress presentations - Paper “check for axes” XIII PMT meeting – October.
Concept of Measurement
Evaluation Research COMT 502. topics Evaluation research –Process of determining whether intervention has its intended result.
12-Apr CSCE790T Medical Image Processing University of South Carolina Department of Computer Science 3D Active Shape Models Integrating Robust Edge.
RESEARCH METHODS Lecture 19
Sampling and Data Collection
Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort 1 th Project Management Team Meeting August 31, 2011.
Model-based Automatic AC/PC Detection on Three-dimensional MRI Scans Babak A. Ardekani, Ph.D., Alvin H. Bachman, Ph.D., Ali Tabesh, Ph.D. The Nathan S.
TGDC Meeting, July 2011 Usability and Accessibility Test Methods: Preliminary Findings on Validation Sharon Laskowski, Ph.D. Manager, NIST Visualization.
Mapping the correlations between CSF Abeta and tau and hippocampal atrophy in 282 ADNI subjects Liana G. Apostolova, Amity E. Green, Kristy S. Hwang, Jonathan.
Copyright © 2008 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 12 Undertaking Research for Specific Purposes.
Clinical Social Work Research Patience Matute-Ewelisane Eugene Shabash Jayne Griffin.
Research Methodology For IB Psychology Students. Empirical Investigation The collecting of objective information firsthand, by making careful measurements.
Agenda: - Axes check - Validation vs Pathology - Publication Policy - Paper Publications - Congress presentations & Meetings VIII PMT meeting – Mar 21,
Usability testing. Goals & questions focus on how well users perform tasks with the product. – typical users – doing typical tasks. Comparison of products.
AAN –Toronto, April 11, 2010 Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort INTRODUCTION.
J OURNAL C LUB : Cardoso et al., University College London, UK “STEPS: Similarity and Truth Estimation for Propagated Segmentations and its application.
Background Management Council (MC) was briefed on approach in early Feb 2003 and approved it Agreed that every Service Group (SG) will participate in.
MRI as a Potential Surrogate Marker in the ADCS MCI Trial
Psychometrics & Validation Psychometrics & Measurement Validity Properties of a “good measure” –Standardization –Reliability –Validity A Taxonomy of Item.
Evidence-based Education and the Culture of Special Education Chair: Jack States, Wing Institute Discussant: Teri Palmer, University of Oregon.
Wei Chen CCNI Journal Club Alzheimer’s disease (AD): imaging & cognition imaging & cognition.
VΙΙ Meeting of the Expert Group on Demand Reduction, September 13 – 15, 2005, Ottawa, Canada Data Considerations for Prevention Programme Performance Indicators.
AAN –Toronto, April 11, 2010 Development of a harmonized protocol for hippocampal tracing An EADC-ADNI joint effort METHODS.
Universiteit Antwerpen Conference "New Frontiers in Evaluation", Vienna, April 24th-25th Reliability and Comparability of Peer Review Results Nadine.
NA-MIC National Alliance for Medical Image Computing Mental Illness and Neuroscience Discovery (MIND) Consortium Data Randy L. Gollub,
KVEC Presents PGES Observation Calibration Are You On Target?
An Expanded Model of Evidence-based Practice in Special Education Randy Keyworth Jack States Ronnie Detrich Wing Institute.
Correction of bias field artifact in T1w MR images of the thigh and calf muscles Correction de contraste d’images IRM Projet A.
References [1] Coupé et al., An optimized blockwise nonlocal means denoising filter for 3-D magnetic resonance images. IEEE TMI, 27(4):425–441, 2008.
COMPARATIVE LATERALIZING ABILITY of MULTIMODALITY MR IMAGING in TEMPORAL LOBE EPILEPSY ¹ Karabekir Ercan, M.D. ¹ ¹ H.Pinar Gunbey, M.D. ¹ ¹ Elcin Zan,
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 11 Specific Types of Quantitative Research.
Accuracy, Reliability, and Validity of Freesurfer Measurements David H. Salat
Maguire Physiological Psychology The Core Studies.
Novel method of 3D-assisted quantitative assessment of orbital volume using an open-source software platform 1 Victor Bong-Hang Shyu, M.D., 1 Chih- hao.
DECIDE: "Scientific and Clinical Perspectives“: Claudio Babiloni (UNIFG) and Giovanni Frisoni (IRCCS Brescia)
MASTER'S THESIS SEMINAR DR. SHUAIQIANG WANG DEPARTMENT OF CS-IS, JYU.
Impact of Intersection Angle on Safety
NATA Foundation Student Grants Process
TJTS505: Master's Thesis Seminar
Understanding Quantitative Research
AXIS critical Appraisal of cross sectional Studies
Title: Validating a theoretical framework for describing computer programming processes 29 November 2017.
Week 3 Class Discussion.
RELATING NATIONAL EXTERNAL EXAMINATIONS IN SLOVENIA TO THE CEFR LEVELS
Properties of Translations
Properties of Reflections
RESEARCH METHODS Lecture 19
Student Evaluations of Teaching (SETs)
Automating stroke lesion segmentation in brain images using a multi-model multi-path convolutional neural network Yunzhe.
Presentation transcript:

Agenda: - Axes check - Validation vs Pathology - Paper Publications - Congress presentations & Meetings VII PMT meeting – Feb 29, 2012

Project method 1.Delphi is required to take all decisions based on experience + evidence 2.Empirical data on SUs taken on ACPC is fed to Delphi (external & internal landmarks) 3.Delphi decides on which SUs make up harmo hippo (internal landmarks) 4.Delphi decides that best segmentation is on hippo axis ( in favour-neutral-against) 5.Empirical evidence collected that total hippo volume on ACPC = hippo axis

1. Delphi is required to take all decisions based on experience + evidence

2. Empirical data on SUs taken on ACPC is fed to Delphi (external & internal landmarks) 3. Delphi decides on which SUs make up harmo hippo (internal landmarks) Internal Landmarks (n)External Landmarks (n) SUs: Minimum Hippocampus51 Alveus/Fimbria51 Subiculum - Oblique Line51 Subiculum – Horizontal Line51 Subiculum – Morphology51 Tail - Crura42 Tail – Tail End51 Harmonized Hippo60

2. Empirical data on SUs taken on ACPC is fed to Delphi (external & internal landmarks) 3. Delphi decides on which SUs make up harmo hippo (internal landmarks)

HippoAC-PC same Round 1: Which plane Fisher’s exact Round 2: Agree on hippo plane Disagree same Agree Rounds 3 and 4 converged on mean angle between R and L axes Fisher’s Binomial 4. Delphi decides that best segmentation is on hippo axis ( in favour-neutral-against) DATA SHOWN IN PARIS, JULY 2011 Delphi: access with user: prova, pw: provawww.hippocampal-protocol.net/SOPs/delphipanel.html

5. Empirical evidence collected that total hippo volume on ACPC = hippo axis TracerHippo_LeftHippo_Right Intra-rater: same 10 subjects (20 images). different orientation (ACPC vs Hippocampal Axes) MB GP LA* Inter-rater: all 20 images (ACPC+Hippocampal Axes) MB-GP-LA* Inter-rater: only for ACPC orientation (10 images) MB-GP-LA* Inter-rater: only for Hippocampal Axes orientation (10 images) MB-GP-LA LA*= LA traced only 5 images oriented along ACPC line

Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) 5 expert tracers 20 naive tracers The best 5 naive tracers Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) GOLD STANDARD Local Protocol : 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) 1 tracer Qualification (20 tracers) Qualification global and local 95% confidence intervals RM-ANOVA: test of rater and rater by center terms RM-ANOVA: test of main effects side, trace- retrace, atrophy, time, scanner, rater Local Protocol : Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Harmonized Protocol : Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens ’ s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) RM-ANOVA: test of protocol main effect VARIABILITY EVALUATION VALIDATION vs PATHOLOGY Training (tracing 20 hippos on 1.5T ADNI scans with each SU) (SAME on 3T ADNI scans) Delphi panel → harmonized prot VALIDATION ON 1800 HIPPOCAMPI IN TOTAL

Pros & Cons Methodological 1)Violate general method of the project (decisional tree) 2)Interfere with the aim of the project (define gold standard as best possible accuracy) 3)Disregard and contradict the opinion of 20 international experts of hippocampus Methodological 1)Contrast with the orientation on which evidence was gathered Note that this contrast : did not constitute a problem for 20 international hippo experts did not correspond to quantitative evidence showing any impact on the Harmo Prot Accepting the consensus on hippo axes would: Practical Reduce partial volume effects and facilitate manual tracing (tracers and panelists’ subjective reports) Changing to AC-PC would: Practical Facilitate automatic preprocessing

Validation versus pathology Local Protocol : 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) 1 tracer Qualification on benchmark images (total for each rater: 40 hippos) Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) RM-ANOVA: test of protocol main effect WHICH SAMPLE?

Paper Operationalization Alzheimer’s and Dementia

Papers describing the project Survey of protocols (preliminary phase; published, JAD 2011) Operationalization (preliminary phase; to be completed) Delphi consensus (Brescia Team) Master tracers’ practice and reliability (Brescia Team) Development of certification platform (Duchesne and coll?) Validation data and Protocol definition (Brescia Team) Validation vs pathology (TBD)

Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) 5 expert tracers 20 naive tracers The best 5 naive tracers Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) GOLD STANDARD Local Protocol : 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) 1 tracer Qualification (20 tracers) Qualification global and local 95% confidence intervals RM-ANOVA: test of rater and rater by center terms RM-ANOVA: test of main effects side, trace- retrace, atrophy, time, scanner, rater Local Protocol : Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Harmonized Protocol : Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) RM-ANOVA: test of protocol main effect VARIABILITY EVALUATION VALIDATION vs PATHOLOGY Training (tracing 20 hippos on 1.5T ADNI scans with each SU) (SAME on 3T ADNI scans) Delphi panel → harmonized prot

Congress presentations - AAN 2012, New Orleans Oral Presentation S04.003, April 24, Aging and Dementia: Therapeutic Interventions, 1.30 pm M Boccardi, M Bocchetta, L Apostolova, J Barnes, G Bartzokis, G Corbetta, C DeCarli, L DeToledo-Morrell, M Firbank, R Ganzola, L Gerritsen, W Henneman, R Killiany, N Malykhin, P Pasqualetti, J Pruessner, A Redolfi, N Robitaille, H Soininen, D Tolomeo, L Wang, C Watson, H Wolf, S Duchesne, CR Jack, GB Frisoni. Delphi consensus on landmarks for the manual segmentation of the hippocampus on MRI: preliminary results from the EADC- ADNI Harmonized Protocol working group. - AAIC 2012, Vancouver Abstract n , submitted to AAIC 2012 M Boccardi, M Bocchetta, A Redolfi, P Pasqualetti, R Ganzola, N Robitaille, S Duchesne, CR Jack, GB Frisoni and the EADC-ADNI Hippocampal Harmonization Group. Definition of Harmonized Protocol for Hippocampal Segmentation Abstract n , submitted to AIC 2012 M Boccardi, M Bocchetta, A Redolfi, P Pasqualetti, R Ganzola, N Robitaille, S Duchesne, CR Jack, GB Frisoni and the EADC-ADNI Hippocampal Harmonization Group. Definition of Harmonized Protocol for Hippocampal Segmentation

AAN – April 24, 2012 AGENDA: 1. Delphi Paper; 2. Publication policy AAIC – July 2012 AGENDA: 1. Presentation of Harmonized Protocol; 2. Reliability figures SOPs periodical meetings

GANTT