1. Lessons Learned from Measuring Cell Response by Quantitative Automated Microscopy FDA Workshop, Potency Measurements for Cellular and Gene Therapy Products, Feb 2006 John T. Elliott, Alex Tona, Kurt Langenbach and Anne Plant NIST, Biochemical Science Division, Gaithersburg, MD 20899 John T. Elliott, Alex Tona, Kurt Langenbach and Anne Plant NIST, Biochemical Science Division, Gaithersburg, MD 20899

2. NIST Mission Founded in 1901, NIST is a non-regulatory federal agency within the U.S. Department of Commerce NIST's mission: To develop and promote measurement, standards, and technology to enhance productivity, facilitate trade, and improve the quality of life.

3. Using Cells as Measurement Devices Mammalian Cell Extracellular Matrix Nutrients Growth Factors Cell-Cell Interactions Scaffold Materials Inputs Signals Other Factors Topography Mechanical Forces Cell Status Inflammation Proliferation Differentiation Remodeling Apoptosis Biomarkers Protein “X” Tenascin gene Cell morphology Protein “Y” Cell Cycle Progression Cell “Meter” Inputs Signals Output Signals Assay Standards/ Reference Materials Measurement Standards/ Reference Materials

4. Known materials/ conditions Controls Unknown materials/ test conditions Test Cell Response Quantitative Cell Measurement/ Instrumentation Statistics Schematic of a Cell-based Assay Calibration standards Data extraction standards SOP Choice of statistical test Quality specifications SOP Highly controlled environment ± control reference materials SOP=Standard operating procedures Valid biomarker for cell function SOP for cell handling Assay validation SOP Requires Validation Steps

5. Precision, Robustness and Accuracy in Cell-based Measurements Precision – reproducibility in replicates Metric- mean ± SD, CV Robustness – long term reproducibility Metric- variance of a quality factor (i.e. Z-factor, S/N) Accuracy – obtaining correct answer from run Metric- comparison to a certified reference material (i.e. length or fluorescence intensity standards).

6. Expect a distribution of cell responses Cell Shape Gene Activation (TN1-GFP) Single cell clone of NIH3T3-TN1-GFP-fibroblast on TCPS Measuring the distribution of responses provides a more accurate representation of the cell population

7. Automated Fluorescence Microscopy Multi-fluorophore imaging Cell ShapeNucleus3 rd marker Image data is information rich; multiparameter information Requires image analysis to extract data Automated microscopy allows: -Unbiased data collection from a cell population. -Can be less labor intensive than flow cytometry Automated microscopy allows: -Unbiased data collection from a cell population. -Can be less labor intensive than flow cytometry

8. Reference Materials for Calibrating Instruments Validates instrumentation is operating properly (i.e. dynamic range, lamp intensity, and linearity of response.) Fluorescent glass reference materials (+/- control) under development. Calibrating fluorescence microscopes for quantitative cell- based measurements, J. Elliott et al. Under preparation. Length standards Optical Property Standards Chemical Standards SRM Fluorescein solution SRM Flow Cytometry beads SRM Fluorometer (in prep) SRM Fluorescent wavelength NIST Standard Reference Materials (SRM)

9. Distributions and “Mean” value of Cell- based Measurements Cell Area (  m 2 ) Relative GFP Fluorescence Relative Cell Number Gaussian-like Response Distribution Non-Gaussian Response Distribution Cell Morphology Measurement GFP Fluorescence Measurement Specification for reproducibility of replicate means (precision) =647±44  m 2 =73297±8300 Average mean intensity (n=4) Average mean area (n=4) (CV=0.07) (CV=0.11) mean

10. Dependence of Accuracy and Precision on Cell Number. Precision and accuracy of the average mean GFP intensity measurement are influenced by number of cells sampled and distribution shape. Relative GFP Fluorescence Relative Cell Number mean Non-Gaussian Response Distribution Mean intensity from replicates CV of replicate means Cell Number Sampled Accuracy Precision Must use this number of cells in measurement GFP Fluorescence Measurement

11. Setting up a Minimal Assay Processing conditions Control Samples + + + + - - - - replicates p1p2p3p4 p1p2p3p4 p1p2p3p4 p1p2p3p4 Control samples allow validation of biological measurement Replicates allow uncertainty metrics to be determined

12. Z-factor as a Metric for Assay Quality Average means and standard deviations are obtained from positive and negative control replicates. Z-factors can be used to establish an assay robustness specification. (3  - + 3  + ) |m - - m + | 1-Z= Reference: Zhang, et al. (1999) J. Biomol. Screen. 4, 67.

13. Using Z-factor to Evaluate an Assay +Ctrl -Ctrl Cell response +Ctrl -Ctrl Cell response +Ctrl -Ctrl Cell response Dynamic range is larger than confidence interval Sensitivity~1 Specificity~1 Z>0.5 Dynamic range is similar to confidence intervals Sensitivity~0.95 Specificity~0.95 Z=0.5 Dynamic range is smaller than confidence interval Sensitivity<0.95 Specificity<0.95 Z<0.5 (Requires threshold assumptions) Dynamic range

14. Z-factor for a Morphology/Biomaterial Assay Multiple Assays-> Robustness Specification: Z=0.50±0.05 We use a cell morphology assay to ensure quality control of a manufactured cell culture surface. -Ctrl +Ctrl Mean - Mean + Histogram Distributions Cell Size (  m 2 ) Fraction of cells (3  1 + 3  2 ) |m 1 - m 2 | 1-Z= Average Mean + =1686±148 (n=6) Average Mean - =5282±404 (n=5) Z=0.53 (~200 cells/well) -Ctrl +Ctrl Cell area TC polystyreneCollagen films From Replicate Controls: Dynamic range Selectivity~0.95 Specificity~0.95

15. KS Test and the D-Statistic The KS test is a non-parametric test for statistically comparing distributions of data.The KS test is a non-parametric test for statistically comparing distributions of data. The D-statistic is the maximum absolute vertical distance between two cumulative distributions.The D-statistic is the maximum absolute vertical distance between two cumulative distributions. It is sensitive to changes in distribution position and shape.It is sensitive to changes in distribution position and shape. It varies from 0 to 1.It varies from 0 to 1. Prepare Cumulative Distribution D-statistic D=max(abs(c1-c2)) Cell Response Sum number of cells Cell Response Relative # of cells

16. Advantage of using a D-statistic over mean value differences mean 2 mean 1 mean 2 mean 1 Yes (Z~0.5) Measurable Difference? Mean Value D-statistic Cumulative DistributionsResponse Distributions mean 2 mean 1 +Ctrl-Ctrl D D D Cell Response Fraction of cells Sum of cells 0 1 No (Z=0) No (Z=0) Yes (Z~0.5) Yes (Z>0) Yes (Z>0) Reference: Vogt A., et al. (2005) J. Biol. Chem. 280(19) 19078.

17. Statistical Evaluation Statistics can help decide if the observed difference between two measurements is likely to be caused by random chance. Statistics requires measurements with uncertainty values This means having replicate experiments (n>3 recommended) Statistical evaluations are most helpful in deciding if small differences are significant.

18. Summary Cells exhibit a distribution of responses A valid measurement of the distribution of cellular responses requires sampling an adequate number of cells. Internal positive and negative controls during assay measurement can be used to evaluate assay quality and robustness. Alternative methods to measure differences in cell response can take advantage distribution shape information. Statistical analysis requires measurements with uncertainty values. It is most useful for determining the significance of small measurement differences.

20. Fibrillar Films Non-fibrillar Films Relative number of cells Prepared 8.12.02 Relative number of cells Cell Area (  m 2 ) Prepared 8.12.02 Cell Area (  m 2 ) 5 Replicate Films 1 year later 5 Replicate Films 1 year later 5 Replicate Films 1 year later 5 Replicate Films 1 year later Reproducibility of Morphology Results The response distribution is highly reproducible.

21. Using a D-statistic to Measure Changes in Cell Measurements.

22. Native Fibrillar Collagen Thin Films Side View Average 23±2 nm Max. ~400 nm Large fibrils (~200 nm dia, >20  m long) Monomer/Small fibrils (~5 nm dia, <500 nm long) ~100 nm 50  m 5  m AFM Z max =300nm Optical Microscopy

23. Automated Quantitative Microscopy Computer w/image processing software CCD camera Beam splitter Objective Excitation Filter wheel X-Y translation stage Focus motor Emission filter wheel Excitation lamp Computer w/image processing software CCD camera Multi Pass Beam splitter Objective Excitation Filter wheel X-Y translation stage Focus motor Emission filter wheel Excitation lamp Multi-fluorophore imaging Cell ShapeNucleus3 rd marker Advantages: -Unbiased data collection -Sample large number of cells -Multi-fluorophore imaging -Live cell imaging -Evaluate cells in real culture conditions Advantages: -Unbiased data collection -Sample large number of cells -Multi-fluorophore imaging -Live cell imaging -Evaluate cells in real culture conditions