Improved Techniques for Fast Sliding Thin-Slab Volume Visualization Hello Janice Turlington…What to say about me??? How to introduce Bill??? relationship (advisor), title,etc. Improved Techniques for Fast Sliding Thin-Slab Volume Visualization…………………….. Not the correction…NEW techniques!! Janice Z Turlington*, William E. Higgins* ** Electrical Engineering, **The BioEngineering Program, and *Computer Science and Engineering The Pennsylvania State University, University Park, PA 16802 SPIE Medical Imaging 2000, San Diego CA, 13 February

Motivation Objectives Show “true” information Volumetric Computed Tomography (VCT) The Motivation: to find small structures in a volume of CT data… mainly in order to find lymph nodes in the lungs. Today, I will be showing you the two techniques that came out of my research. A depth perspective rendering process and traversal-gradient range process. Though I will not be talking about lymph nodes here…I will say that results that are coming from these techniques are quite exciting and…strongly indicate they are locating the lymph nodes in the lungs. Starting out, one highly important thing is to build a process that will be used in the medical community …so they need information they feel confident and justified to use. Meaning to me….that no matter the quality or quantity that … Processing result in information that is truly in the CT data. So to do this…data integrity has to be safeguarded… Use the CT data directly…do no pre-processing. Use functions that did not bias output. 3. Retain full calculation values throughout procedures (I.e., rounding) Lastly, techniques are for real-time use in the Virtual Navigator system developed by Dr. Higgins research group. This is being presented at this conference. …NEED formal name, Bill???, our research group is developing called??? ….need day Motivation Locate small structures in 3D CT volume Objectives Show “true” information Confident results for professional use Fast techniques for real-time use

Existing Visualization Techniques volume/surface rendering4 multi-planar reconstruction2 Looking at the issue of small structures…I looked at existing techniques Present techniques projection imaging1 2D image resembling a chest X-ray 1{Hohne87,Napel92}… Good global view oblique slice viewing2 (multi-planar reconstruction MPR) 2D cross-sectional data at arbitrary orientations 2{Robb1988,Remy96,McGuinness97} curved-section reformatting3 (“tube” view): 2D straightened arbitrary path 3{Robb1988,Hara96,Ramaswamy99} volume or surface rendering4 global 3D view of structures in a 3D image 4{Ney90,Drebin88,Tiede90} virtual endoscopic rendering5 3D rendering of interior endoluminal structures 5{Vining94,Ramaswamy1999} projection imaging1 curved-section reformatting3 virtual endoscopic rendering5

Existing Limitations dependence on a priori knowledge Present Visual Limitations dependence on a prior knowledge voxel intensity decisions thresholding intensity mapping gradient mapping partial-volume artifacts blending artifacts spatial resolution voxel boundaries (vs. true image interfaces) dependence on a priori knowledge voxel intensity decisions partial-volume artifacts voxel boundaries 1{Hohne87,Napel92} 2{Robb1988,Remy96,McGuinness97} 3{Robb1988,Hara96,Ramaswamy99} 4{Ney90,Drebin88,Tiede90} 5{Vining94,Ramaswamy99}

Sliding Thin-Slab Visualization (STS-MIP) Technique that caught my eye was: Sandy Napel: Sliding Thin-slab visualization Because it removed obstructing tissues…. And extended the length of vessels STS-MIP recognized by Remy-Jardin etc. Clinical Study of the Value of Sliding-thin-slab Maximum Intensity Projection CT scans in the detection of Mild micronodular patterns Radiology 1996 Evaluated STS MIP projection reconstructions in the assessment of micronodular patterns of low profusion in diffuse infiltrative lung disease. Due to increased conspicuity of tiny lesions: 1. expected to improve the understanding of lung changes 2. enabled a more precise characterization of distribution of lung changes than conventional CT scans “Because of the ability to increase the conspicuity of tiny lesions, sliding thin-slab MIP is expected to improve our understanding of lung changes....we found sliding thin-slab MIPs enabled a more precise characterization of distribution of lung changes than conventional CT scans.” Martine Remy-Jardin, Jacques Remy, Dominique Artaud, Franck Deschildre, Alain Duhamel Diffuse Infiltrative Lung Disease: Clinical Value of Sliding-thin-slab Maximum Intensity Projection CT scans in the detection of Mild micronodular patterns Radiology 1996 {Napel 92} improve understanding of lung changes more precise characterization of distribution changes {Remy-Jardin et al. 96}

Depth-Weighted Maximum (DWmax) Depth-Weighted Maximum (DWmax) Our New STS Techniques Depth-Weighted Maximum (DWmax) Extreme Gradient (EG) So we now have two new STS rendering techniques Depth-Weighted Maximum (DWmax) Extreme Gradient (EG) Both Fast general-purpose algorithms Plusssss: you can …Dynamic sequence visualization Dwmax: a striking and accurate 3D impression EG: a 4th dimension of information….the extent of change present within a slab fast general-purpose algorithms dynamic sequence visualization accurate 3D impression slab change volume evolution

Sliding Thin-Slab Movement The sliding thin-slab method windows a shallow (arbitrarily oriented) sub-volume of data Window processed then slid forward Each voxel along a ray is processed individually… according to user defined mappings of voxel intensity voxel gradient Which then defines voxel opacity One of three Functions then composite these opacities along a ray: maximum intensity (MIP) maximum opacity (MOP) classic volume rendering (CVR) volume window column point slab

Fast STS Algorithms temporal coherence1 only 2 slices differ Fast methods apply for ALL methods: Dwmax, EG, Max, Min Reason: incorporate temporal coherence. Concept: if one's point of view varies slowly, then consecutive views change little. With Window processing of sub-volumes of data 2.) windowed volume of data for a slab differs only by two slices from the windowed volume of data for the slab that follows. 3.) many of the computations performed for a slab can be applied to next slab Two techniques were developed…Depth-Weighted Maximum And EG Lets look at Dwmax modeling… After this, say you’ll now discuss Dwmax and EG, and fast algorithms will come out here temporal coherence1 adjacent views change little only 2 slices differ slabi window calculations apply to slab i+1 1{Ramaswamy1999}

Depth-Weighted Maximum (DWmax) Was there other information in the windowed data??? Some changes: 1.) to have the best chance to get small structures work directly with actual CT scan data….and keep it clean and unaltered So must: remove intensity biasing…that happens in thresholding, intensity mapping, gradient mapping… all takes out small detail small structures!!! Which meant A. take human preprocessing decisions out of the loop B. Determine ray values using general decisions based on relative perspective This went right along with the modeling of DWmax Model to achieve goalsEnvision sight as a group of progressive planes… natural visual characteristics Closest plane is most most intense… Remaining planes fade into the distance till perception is lost. Vision is defined by maximum opaque object encountered, before vision is lost to distance. Global factor, such as the side of building, may stop planes shy of farthest visible plane. For a small distance, projection of view is parallel (like ideal X-ray directed normal to viewing plane). Now to take it to Dwmax… Planes = Image slices Closet plane = Base slice first slice in processing window (full intensity) Distance perception is lost = Depth of Vision (dv) Distance building is encountered = Field of View (ds) depth of processing window. Fading = Depth-Weighting function of dv and ds Plane at distance of building = End slice in window = last slice processed Therefore all weighting is done at the slice level and voxels do not need to be individually handled…main factor for fast processing wi = (depth of view – distancei from base)

Depth-Weighting: 1D process Depth weighting…so at each voxel…weighting determined by slice level is blindly applied For each voxel in the base slice Column of window data is processed From base out Base slice given full intensity…weight =1 Slices away from base, decreased by number of slice units they are away from base All normalized by depth of vision Only data up to the field of view is processed Maximum weighted voxel values becomes slab point value Temporal coherence: Move window up ….new weighted end slice is larger than previous slab, becomes present slab value If previous base slice was previous slab max, must brute force present slab If neither end defines process….up-weight previous slab to present slab Window defines slab data Depth weights not dependent on voxel intensity nor neighbors Weighting is a slice level process….a function of slice’s location in window weight independent of (x,y) voxel location in slice….same weight for every voxels in slice… SPEED ADVATAGE Only calculate once then apply to all windows Set of depth-weights apply to all windows Why Depth of vision??? Advantages of depth of vision A. Extending vision beyond the field of view provides a relative distance that effectively varies like-tissue intensities on the different slice to produce depth perceptive view. B. allows end slice in window to retain values that add legitimate definition to slab..without last slice looses all its ablity to add to information to slab.

What you can see it is… because of the slice-level weighting DWmax Algorithms volume sub-volume Here is the complete code for Dwmax…. Paper has complete detail Too much to go into now… it is addressed in the conference proceedings, and I will be glad to discuss it after the presentation What you can see it is… because of the slice-level weighting Quite Condensed Simplified Functions are not 3D… volume level: 1D direction of movement, window level: 2D slice plane, column level (voxel level) 1D direction of movement 3. Point Functions only have one decision statement column .

STS-MIP DWmax CT image slice {Napel 1992} {Turlington 2000} Healthy Human 3D electron-beam CT scan (no contrast agent) 3mm/0.781mm/0.781mm Transvere plane CT slice view STSMIP Extension of vessels BUT: NO impression of depth or 3D overlapping causes limited use of additional structural length information Only usable as extra or secondary tool DWmax 3D local structure Strong impression of Depth Endoluminal information STS-MIP DWmax {Napel 1992} {Turlington 2000}

DWmax: Transverse Sequence Views Healthy human: 0.781/0.781mm Effective view of physical structure of interior tissues, clear visual perspective into thoracic cavity External and internal relationship of structures Peer into airways and see endoluminal structure s=43 s=49

coronal detail DWmax: coronal sequence views projection image No preprocessing, no segmentation, no contrast agent. Note soft tissue definition and tracheal detail…extension of vessels…clarity and extension of information 3mm/.0781mm resolution vessels, airways, soft tissue visible, traversal through front to back Ds=19 Dv=30 projection image coronal sequence views single slice

DWmax: sagittal detail *Can see very helpful in particular cases for close detailed study but especially great for general viewing purposes…initial look at CT *Depth perception without computation and memory cost *Fast, real-time worthy…slice level weighting means low computation and minimual memory required) *Packages large quantiy of CT images for dynamic viewing *No knowlege of image required…general process *Only 2 slab size input parameters (unrelated to intensity characteristics) *Slabs contain the entire full range of image data…so only single run per slab size is necessary Dwmax: Easy & General depth perception and interior information fast real-time processing1 general process no specific data type required no knowledge of image needed only 2 input parameters Practical>>>efficient packaging for large quantity of data New helical multi-detector scanner highly increases number of images highly effective when dynamically viewed Striking or abnormal changes “jump out” like beacons…grab one’s immediate attention Quick and effective recognition entire full range of image data is available Only one run for all needed information Bonus similarity to cross-sectional imaging minimal learning curve + direct integration of experience (Note Naidich comments) base slice base slice

Mapping of max tissue-density changes in slab Extreme Gradient (EG) Mapping of max tissue-density changes in slab Extreme Gradient A mapping of the maximum changes in tissue density occurring within the depth of a slab. An EG point = magnitude of the difference between maximum - minimum intensity in the slices within a STS window. EG a function of image extremes, structure size, and slab size s(x,y) = max x,y{ } – max x,y{ }

volume sub-volume column EG Algorithms EG Characteristics Tissues fall into well-separated unique bins (inherent from distinct separation between tissue intensities). tissue value are consistent over a large range of slab sizes (because function of extremes). boarders outline pattern change structural size change (slab level) shape changes within a slab (slab level) evolutional change in a volume. Shape/intensity change proportional to value/pattern So Radical change in structure or intensity easily seen in EG value and pattern change Entire range of image change is contained in a slab column

Extreme Gradient large change EG not windowed Function of: Extreme Gradient contains all change information Depending on the tissue of interest or level of changes being studied An EG slab can be windowed to show this EG: 4th dimension of DWmax study tissue dynamics traverse volume consecutive slab (volume evolution) hold base vary slab size (detect entrace of details) pattern boundaries structural size and shape changes within a slab evolutional change in a volume. detect smaller structures hidden or buried in DWmax detect sudden or irregular grow The top view shows EG slide viewed with normal gray value display map Window for low values, interior lung tissue and steady bone is shown most intensely. Window for moderate values, highlights the change in heart size and vessles introduced Window for large values, captures the branching of a bronchus in this slab, rib bones curving When viewing Dwmax, this EG feature would be useful to filter the Dwmax view i.e., soft tissue if focus, or small vessle location is the focus… EG: Consistent & distinct values Practical applications: focused-dynamic viewing (when viewing DWmax display points windowed using EG, hold all other points constant) tissue identification structure segmentation image extremes structure size slab size small change moderate change EG not windowed large change

Endoluminal Border Detection Pratical application for combination of techniques Edge functionality of EG + DWmax depth Parallel ray processing eliminates perspective concerns Overlay EG onto Dwmax:…DETECT Wall thickness Tissue irregularities Abnormal growth (seen here) This slide: Human lung caner case: Vary size of EG and keep Constant Dwmax… outline of EG show details of trachea changes at varying depths Ds=10, Dv=20 Shallow EG = 4…same size EG=10 merging DWmax and EG

Computation Gains of New Techniques inversely proportionally to slab-size (ds)

Current Reconstruction Techniques {Naidich et al. 97} cross-sectional imaging is CT gold standard axial images alone enable remarkably accurate assessment true paradigm shift requires comparison to traditional methods David Naidich James Gruden Georgeann McGuinness Dorothy McCauley Meenaskshi Bhalla Journal of Thoracic Imaging 1997 Volumetric (Helical/spiral) CT (VCT) of the airways purpose of the paper…review various reconstruction techniques, and based on current knowledge, place them in a appropriate clinical context. “At present, cross-sectional imaging remains the CT ‘gold standard’ for assessing pathology of the airways and is clearly complimentary to fiberoptic bronchoscopy” “Despite the limitations outlined herein, axial images alone enable remarkably accurate assessment of the central and peripheral airways… Any attempt to achieve a true ‘paradigm shift’ will require comparison of these new techniques to the traditional methods of airway evaluation.”

currently under development lymph node detection S=39

DWmax EG: soft tissue lymph node? EG: high density

Conclusion: New Techniques fast, real-time small structures visible depth perception true information fast, real-time worthy slice level weighting standard axis directed processing + parallel processing temporal coherence confidence similar to cross-sectional imaging companions technique compliment slab information justification interior details detection of non-visible structures evolution information visual depth perception small structure detail full relative view PLUS: EG definition and detection true information from safeguarded data intensity-independent weighting Or (x,y) independent weighting (location of voxel in slice) No parallel ray processing + standard-axis directed processing

My thanks to: Dr. William E. Higgins The Whitaker Foundation Dr. K. Kirk Shung Dr. Cheng Dong My Father “For nothing is hidden, except to be revealed; nor has been secret, but that it should come to light.” Mark 4:22

Virtual Bronchoscopic Approach Combining 3D CT and Endoscopic Video 2:40 pm Today…California Room A. J. Sherbondy Virtual Bronchoscopic Approach Combining 3D CT and Endoscopic Video Physiology and Functions from Multidimensional Images Conference 3978 5:30 to 7:30 pm Today…California Room G. McLennan The Place of Virtual Bronchoscopy in Clinical Practice: Barriers and Solutions

Occurs in less than 2%of points DWmax error NOT visible Occurs in less than 2%of points must occur simultaneously: closer to base than voxel of previous slab’s max depth-weighted voxel must differ: < 5.5% of 16-bit previous-slab point (=0.7% 8-bit) if error occurs: errror < 5 HU (16-bit) or 0 HU (8-bit)