1. Analyze User Instructions, adapted from MVT Bob Schwanke Siemens WITH FUNDING SUPPORT PROVIDED BY NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

2. QI-Bench Overview 2222 Formulate Reference Data Sets QIBO Specify RDF Triple Store CT Volumetry CT obtained_by Tumor growth measure_of Therapeutic Efficacy Therapeutic Efficacy used_for Analyze Y=β 0..n +β 1 (QIB)+β 2 T+ e ij Execute Feedbac k

3. BiomarkerDB.SummaryStatistic+ = Analyze ({ReferenceDataSet.CollectedValue}); 33 RoleUse CaseSupported NowGap vs. Model Domain expert Run and modify statistical analyses Web App (thin) n/aCreate the web app Informaticist Configure set of analysis scripts in toolbox Desktop (thick) Run relevant calculations at technical performance level Persist statistical calculations results as N- ary relations in knowledge store, across larger data sets and at clinical level too IT systems expert Configure data input and output services Server-side Connects to (some)caGrid data services Connect directly to RDSM and knowledge store (and through Formulate, broader set of data services)

4. Load Cases and Run MVT (measurement variability tool)

5. AIM output for selected case (from Batch Analysis Service)

6. Choose Analysis Type (we’ll be adding new Types)

7. Ground truth can be direct or indirect

8. Deselect RECIST and WHO measurements Choose Biomarker Measurements to Analyze (we’ll be adding Measurements)

9. R Script for summary statistic Some Methods exist; we will create new ones (e.g., factor analysis)

10. Column per measurement Columns for calculation according to the Methods

11. Intra- and Inter-reader comparisons Output of Analysis Type run (this example intra- reader)

12. Inter-reader measurements Output of Analysis Type run (this example inter- reader)

13. … calculated Additional outputs based on Methods

14. Outliers list Outlier Analysis with Drill-down is Supported (access to raw imagery)

15. Configure plots as needed Resulting plots – we’ll be adding plot types, e.g., box, radar, and K-M

16. 16