1. Metrology in ISO and VESA Standardization for FPDs Paul A. Boynton and Edward F. Kelley NIST (Bldg. 225 Rm. A53) 100 Bureau Dr., Stop 8114 Gaithersburg, MD 20899-8114 www.fpd.nist.gov NOTE: Certain commercial equipment, instruments, materials, systems, and trade names are identified in this paper in order to specify or identify technologies adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the systems or products identified are necessarily the best available for the purpose. IDIM 2003 Daegu, Korea July 10, 2003

2. Robust — Insensitive to small changes in the measurement apparatus that will affect the ease with which reproducibility is attained. Unambiguous — The method is clearly stated and easily understood. Important details that are required for success are not left out. Honest — The measurement method is not devised to hide an obvious deficiency; redefining familiar terms to cloak a problem. Reproducible — Everybody can get the same results on the same display using appropriate instrumentation. This is probably the most important goal of metrology. Extensible — Applicable to as many different technologies as possible permitting intercomparisons of technologies. Distinct — The name of a measurement method must be chosen so that it is not confused with another metric. Measurement methods used in standards and industry need to be…` GOOD DISPLAY METROLOGY—WHAT IS IT?

3. ...and we might be tempted to add (sometimes the following are not possible)... Meaningful — Properly captures the visual experience for task and environment. Measuring what the eye appreciates should not be sacrificed for some related esoteric measurement method of limited use. Accessible — Requiring the use of unusual, highly specialized, or otherwise arcane apparatus or methods would be avoided unless it is necessary (e.g., for people who influence written standards in order to sell their apparatus when it is not necessary to do so). Simple — Procedures should be made as uncomplicated as possible, avoiding deliberate obscuration from elitism or exploitation (e.g., deliberately making the standard so difficult to use that only a few experts can use it). Accommodating — We want to enable as broad a range of apparatus as possible. Example: Character contrast of a black letter on a white screen must first be measured correctly accounting for veiling glare in the measurement apparatus (discussed later). (Calculating that contrast on the basis of the Gaussian beam profile of an electron beam in a CRT to avoid measuring the real contrast would not be a measurement of the contrast; besides it would only be applicable to CRTs.) After a proper contrast measurement is made, if need be, an appropriate vision model can be applied to determine how the eye appreciates that contrast.

4. Ergonomics and vision science must be based upon good metrology – “Measure twice, cut once.” It would be sad to set a compliance standard for a minimum contrast if that contrast was measured incorrectly during the research. Why do we need good display metrology? Between FPDs within a technology – Which LCD is better for my purposes? Level Playing Field — Competition Specification Language Defined Measuring What the Eye Sees Between different technologies of FPDs – How do we compare a plasma display with a front-projection display (or HMD)? Task-dependent specifications possible – How do the contrast requirements differ for a display used in a dark room vs. a display used in a bright surround? Enables clarity and removes ambiguity – Why should we use darkroom contrast vs. ambient contrast vs. highlight contrast? Clear and understandable measurement standards

5. Complications: Not So Simple!!! Eye response quasi-logarithmic yet measurement comparisons linearly based, so we don’t see some of the problems we are measuring. Display output can drift in time both in color and luminance. Photometry-colorimetry uncertainties START at 1 % level. Anticipated luminance uncertainties in the field can be from 2 % to 4 %. Be very happy if you get 1 %!

6. Proper Setup Depends Upon Display Task. A face may show up problems that are difficult to see with other patterns simply because our vision processing is very sensitive to faces. Available: http://www.fpdl.nist.gov/patterns.html as NISTSU*.*http://www.fpdl.nist.gov/patterns.html How will the display be used? Might also try a face as well as a scene, if appropriate to task. What environment (ambient, surround)? Are there manufacturing setup specifications? Gray scales near black and near white are often useful, but may not be sufficient. [FPDM 301-3A] TASK DEPENDENT SETUP

7. During the warm-up of the display is a good time to examine the display for defects and problems. Try out many different patterns and images suitable to the intended display task. [FPDM 301-2D] During a series of measurements the task-specific setup conditions should not be changed to improve any single measurement, unless the task calls for such changes. [FPDM 301- 2E, 305-3] For examples of patterns see: http://www.fpdl.nist.gov/patterns.htmlhttp://www.fpdl.nist.gov/patterns.html This has to do with honesty: We want to measure the display the way it will be used. Unless the task calls for tweaking the display for various needs, the controls should be set after the display is fully warmed up and all measurements should be made at that setting. Warm-Up Time May Be Needed Setup Conditions Remaining Fixed

8. Often Ignored Diagnostics—A Wise Choice! Ostrich effect — we don’t want to know if we have problems. Diagnostics!? What do we need diagnostics for?! We know what we're doing! Not a clue!!! Time consuming — slow us down! We can’t afford the extra equipment to do them. Confidence in apparatus, personnel, and procedures. Problems revealed early. Awareness of how simple things can go wrong. Benefits DIAGNOSTICS

9. Image Photopic Filter Complex Lens Shutter Object Iris CCD with Cover Glass OriginalVeiling GlareLens Flare Reflection between lens surfaces Reflection off of internal lens structure Stray Light Within Detector – Veiling Glare

10. Example: Checkerboard Displayed on Laptop Scientific-grade CCD, TEC, 16-bit (approx.), using a good 35 mm camera lens.

11. 1.5° Subtense with Mask 15° Subtense without Mask Increase in measured luminance with mask removed: Instrument #10.4 % Instrument #21.3 % Instrument #34.8 % Measurement of Full-Screen White Comparison of two identical luminances having different angular sizes. Same screen with & without mask (1.5° or 15° vertical angular diameter of white area from lens of detector) Veiling Glare Can Affect Full-Screen Measurements

12. Measurement of Black Rectangle on White This shows how important it is to anticipate veiling glare in the detection system. Same screen with & without mask (1.5° mask hole, 15° vertical angular diameter of white area from lens of detector) Increase in measured luminance with mask removed: Instrument #1 50 % Instrument #2 325 % Instrument #31200 % Veiling Glare Can Affect Box Measurements

13. With some very high contrast displays, the back reflections off flat masks can cause substantial errors. DETECTOR VIEW USING FRUSTUM Detector Frustum aperture Detector aperture Side view with flat maskSide view with frustum mask Use of Masks — Flat and Frustum

14. 1. SPECULAR (producing a distinct virtual image of the source) 2. & 3. DIFFUSE (has TWO components): 2. LAMBERTIAN (like matte paint) & 3. HAZE (fuzzy ball in specular direction) Three-Component Model of Reflection Reflectance can be thought of as having three components:  =  L +  S +  H, where the diffuse reflectance is  d =  L +  H. ROBUSTNESS

15. Three Component Reflection Model, Cont. Specular Only Haze Only Lambertian Only All Three Virtual Image (if there is a specular component) Specular, Lambertian, Haze

16. BRDF — Three Components: Bidirectional Reflectance Distribution Function A generalization of L = qE. dE, element of illuminance B = D L + S + D H Boring Alert We will drop the wavelength and polarization p dependence. Three Component Reflection Model, Cont.

17. The luminance of the specular component remains constant with change in distance. The peak of the haze changes with distance (according to illuminance). Because the haze peak adds to the specular image, it can appear that the specular image is changing its luminance, but that is not the case. Look carefully at the specular image, you will see it is not changing its luminance when you separate the specular image from the haze peak. Like the Lambertian component, the haze is proportional to the illuminance; but like the specular component, it follows the specular direction. Three Component Reflection Model, Cont.

18. Observed Luminance = Lambertian Component Specular Component + Haze Component + Background gray Distinct image Fuzzy ball Three Component Reflection Model, Cont.

19. Simple BRDF (In Plane) Extremes: Lambertian (flat) Specular (spike) Haze is in between. Haze characteristics: Proportional to illuminance Directed in specular direction NOTE: 3 to 5 orders of magnitude possible (or more!—your eye has no trouble seeing this range!) Detector Source  = 5° Shown FPD Sample    i < 0  i > 0 Three Component Reflection Model, Cont.

21. Much happens within 1° Three Component Reflection Model, Cont.

22. Three components in BRDF often seen in CRTs Three Component Reflection Model, Cont.

23. In the most general case, when there is a Lambertian, specular, and haze component, there are at least four parameters that are needed to specify the reflection characteristics since haze has a peak and a width (at the very least). If we only make one simple measurement or two, the problem is underdetermined and an infinite number of displays can measure the same and look different to the eye! This underscores the need to make several different measurements to adequately characterize reflection from displays. KEY POINT Note Three Component Reflection Model, Cont.

24. With Haze, Measurements Can Be Sensitive to the Geometry of the Apparatus... Lens diameter Focus Source size Source distance …? Haze Reflection Need Not Be Symmetrical. Star patterns and spikes further complicate a full characterization of reflection, accomplished only via a complete BRDF. Example: ± 1° misalignment of apparatus can result in 30% errors in measured reflected luminance. Three Component Reflection Model, Cont.

25. Specular and Lambertian components are easy to measure, but whenever haze is non-trivial, the measurement result can become very sensitive to small perturbations in the apparatus, affecting reproducibility and robustness. HAZE COMPONENT COMPLICATES REFLECTION MEASUREMENTS! Diameter of acceptance area of detector can affect luminance measurement. Luminance meters are calibrated against a uniform source (Lambertian). Three Component Reflection Model, Cont.

26. Small Specular Source Very sensitive to apparatus settings. Looking in the direction of greatest changes — peak region of BRDF, haze contributions seriously affect results. Not robust, except for simplest reflection properties, i.e., without haze—even then can have problems. log 10 BRDF Angle Relative to Detector View

27. Large Specular Source Works with haze because ‘when one side goes up the other side goes down.’ Might improve by using a larger source subtense (angular aperture) than 15°. Captures specular, haze peak, and much of haze profile. Fairly robust, but angles may be a problem. Focus on display is sufficient. log 10 BRDF Angle Relative to Detector View

28. Ring Light Source  r = 20° Avoids haze peak and specular. Captures the tail of the haze (depending upon width) and some contributions from the Lambertian. Fairly robust, but distances may be a problem as might measurement field angle. Works with haze because ‘when one side goes up the other goes down.’ May benefit from using larger source subtense angle. log 10 BRDF Angle Relative to Detector View

29. Diffuse Source Used two configurations: Display inside large integrating sphere Sampling sphere on display surface  d/  = 0.0557 (± 0.3 %) over range  d = 6° to  d = 20°  d/  = 0.0554 (± 3 %) — (exit port soft, plastic foam) This is an example of a robust method. Insensitivity to geometry and apparatus! The disadvantage is that it combines all three reflection components together. Fiber-optic input provides illumination of interior log 10 BRDF Angle Relative to Detector View

30. Most Robust Apparatus VERY ROBUST: Robust in parameter settings AND apparatus employed. Diffuse Source: People don’t like it because it is tough on their displays, gives low contrast values. Results scalable to many diffuse environmental lighting levels. Exploits and combines all three reflection components, Lambertian, specular, and haze. Many say, “not practical,” “not realistic,”…. NOT TRUE! Not so uncommon after all: Beach on a cloudy day Snow field on a cloudy day Light living room with light furniture and even illumination Bubble helicopter in cloud Horizontally held PDA outside on a cloudy day…

31. Large Specular Source Rather robust. Exploits and combines the peak of the haze component and the specular component as well as adding in some of the Lambertian component. ? More work is needed to determine if there are better settings (e.g., larger source subtense) that will secure more robustness—perhaps two settings. A larger subtense approaches diffuse. A smaller subtense captures more of the haze peak and specular with possible loss of robustness.

32. Variable Radius Source Method (VRSM) Isolation of Specular (z) and Haze Peak (h) Method Under Research L LSLS r r Apparatus geometry may be important for extracting any haze-peak information. Focusing on source rather than display may be required for specular component measurement. More work needed. Non-smooth surfaces (e.g., orange- peel like or heat- laminated) or specular- only displays that exhibit spikes may create problems for the method. Can also get haze peak h from curvature near r = 0. LsLs

33. NOTE: These methods integrate about the normal (  axis) or specular direction thus integrating the effects of complicated haze or specular reflection profiles (spikes and stars, etc.). Some displays are entering the arena that are not smooth or where all reflection surfaces may not be primarily from a single plane. These methods accommodate such displays. Diffuse Source Large Specular Source CANDIDATES FOR ROBUST METHODS Note

34. If you have a gut feeling that something is wrong, it probably is. On the other hand, if everything seems perfect, it probably isn’t. Intuition (Gut Feelings!)? CONCLUDING COMMENTS

35. P 12 —Lest We Forget Working at the Bench... Perpetrating paperwork, poppycock, plus protocol paralyzes promising project progress producing poor products. Don’t over document and then under document! Don’t spend so much time documenting untested and preliminary apparatus and data so that you can’t finish the measurement—it’s like polishing garbage. Take the time to document thoroughly after it is working properly. Look for problems, be suspicious. A bright display can light up a dark room, are you measuring the reflection of your white shirt or the side of a lightly-colored instrument (or wall) along with the screen color? How about equipment lights and displays in the room, do they reflect in the screen being measured? Look and see. Don’t assume. If you can see it, the instrument might be affected by it.

36. Whom do you trust? Don't trust anything or anyone (as much as possible), always try to verify things you are tempted to assume, prove to yourself that everything is working properly and that you are not making inappropriate assumptions.

37. Examples: (1) Eye sees high contrast, instrument measures low contrast—trust your eye. (2) Reflections from nearby objects? (3) Light from instrument indication illumination? Look at what your instrument is seeing from its perspective. Trust Your Eye

38. How was that “limitation” of the eye determined? If the instrumentation used to determine the “rule” is not as good as the eye, then what can’t see, the eye or the instrument? What measurements were made? Was the instrumentation capable of an accurate measurement, how do we know? What diagnostics were performed to prove that the instrument was measuring things correctly? How was “adequately” defined? Be a skeptic! We should not compromise good metrology in favor of tradition when that tradition might be based upon inadequate metrology. What Is “Good Enough”?

39. VESA FPDM — Flat Panel Display Measurements Standard Features: Specification of good metrology for displays Self-contained measurement procedures Buffet of measurements—use what you need Easy to use and read Extensible—more will be added as needed Adaptable—affords a variety of equipment Accommodating—special needs permitted Metrology Section, Technical Discussions Section Includes diagnostics, cautions and hints A reasonably priced document ($40) of over 320 pages— VESA 408-957-9270 www.vesa.org Comprehensive Document Available

40. ISO TC159/SC4/WG2 — Visual Display Requirements Under Development Features: Easier to use and read Separate sections for ergonomic requirements, test methods, and compliance routes Specification of good metrology for displays Combines ISO 13406-2 and 9241-3, 7 & 8 Scope to be extended to include applications beyond office environment and technologies other than desktop CRTs and LCDs Extendable—more will be added as needed ISO Grand Revision

41. Courses & Services at NIST http://physics.nist.gov/Divisions/Div844/facilities/photo/Photometry_Course.html NIST Photometry Short Course http://physics.nist.gov/Divisions/Div844/events/srsc.html NIST Spectroradiometry Short Course http://physics.nist.gov/Divisions/Div844/facilities/photo/Projects/colorimetry_of_displays.htm NIST Colorimetry of Displays — A Calibration Facility Based Upon the Four-Color Matrix Method for Correction of Tristimulus Colorimeters NIST Display Metrology Short Course Coming soon (hands on lab work)… if this course goes well… see http://www.fpdl.nist.govhttp://www.fpdl.nist.gov or http://www.fpd.nist.govhttp://www.fpd.nist.gov NIST Flat Panel Display Laboratory — Publications, Links, Overview http://www.fpdl.nist.govhttp://www.fpdl.nist.gov (“FPDL”) or http://www.fpd.nist.govhttp://www.fpd.nist.gov Slides of NIST seminar on metrology: http://www.fpdl.nist.gov/seminars.htmlhttp://www.fpdl.nist.gov/seminars.html