CONFIDENTIAL Phase Profilometry How to measure (small parts of) the world in real time 20 April 2015

CONFIDENTIAL We are Hiring 3D Vision Research Engineer CyberOptics is a leader in 3D sensing solutions for wide range of challenging industrial applications such as semiconductor packaging inspection, high speed robotic assembly, and reverse engineering. We are looking for a 3D computer vision engineer to join our talented team to develop innovative algorithms for CyberOptics high precision products. The ideal candidate has very strong math skills and energizes off of solving practical, real world problems. Our work pushes the state of the art in many areas, principally speed, and measurement performance. Our object detection and classification algorithms must work correctly % of the time. Fortunately, we also design the 3D sensing hardware and control much of the environment. Otherwise, this would be an impossible task. This level of accuracy is far outside the mainstream literature and is the area we operate every day. Major Responsibilities: Develop vision processing requirements for new optical sensing applications in 3D imaging, inspection, alignment and metrology Develop and implement practical and technically innovative computer vision solutions Lead technical efforts in developing and testing enhancements to existing computer vision algorithms Education Required: PhD degree in Computer Science, Electrical Engineering, Mathematics, Physics, or related areas Formal education in computer vision and digital signal processing Experience Required: Demonstrated creativity through computer vision research and industrial experience Proficient in C++ or C# programming Broad understanding of standard mathematical concepts, such as multivariate calculus, numerical solutions of differential equations, linear algebra, statistical hypothesis testing, statistical noise models, and probabilistic modelling Understanding of point cloud processing, object detection, object classification, segmentation, image mosaicking, active and semi- supervised learning, and image filtering techniques Experience taking a mathematical algorithm or idea, applying it to real world problems, and addressing the vagaries present in real world data Ability to rapidly prototype and test mathematical/algorithmic ideas against simulated and stored data sets 3D computer vision experience desired Experience with real time image processing desired

CONFIDENTIAL Cyber Optics About: Publicly traded since 1987 Founded by former UofMN optics professor $40 Million / year revenue ~30 R&D engineers – 4 PhD’s (Math, Electronics, Physics and Machine Vision)

CONFIDENTIAL What is Cyber Optics

CONFIDENTIAL Who am I? PhD Mathematics U of MN (Probability Theory) Bert Fristedt Advisor. Thesis topic Random Walks in Random Environments. Jobs – Secure Computing Corporation: Mathematician / Systems Engineer / Program Manager, – Lockheed Martin Corporation: Mathematician / Systems Engineer: – Cyber Optics : Mathematician

CONFIDENTIAL 3D Measurement 2004 – DARPA Challenge. Best participant, 15 miles out of – 5 Teams complete challenge. What was the enabling technology? – LADAR (a 3D sensor, sensing their environment)

CONFIDENTIAL Kinect

CONFIDENTIAL HP Sprout Consumer PC that will take 3D scans of anything (that isn’t shiny)

CONFIDENTIAL How to measure 3D? Time of Flight (LIDAR) Passive Stereo (Trnio, Photosynth) Profilometry

CONFIDENTIAL What is Phase Profilometry? Step 1) Profilometry Target Laser Receiver θ Foundation of Cyber Optics first product, PRS 30 height measurements / second 9 bits / measurement. Roughly 300 bits / second.

CONFIDENTIAL What is Phase Profilometry? Step 1) Profilometry Note: if I calibrate my laser and receiver, I’m doing 3D vector math: Target Laser Receiver

CONFIDENTIAL Step 2) Linear Profilometry Use a line of light, and a camera….

CONFIDENTIAL Step 3) Structured Light Profilometry Target Receiver Light Projector Let’s parallelize this: Take projector, and do step 1) for each pixel in the projector… Somehow, we need to encode projector position onto target…

CONFIDENTIAL Gray Codes to the rescue

CONFIDENTIAL But, there’s a problem: Projection systems have blur…

CONFIDENTIAL Finally: Phase Profilometry An optically blurred sinusoid is still a sinusoid. – Optics act as a convolution on the transmitted signal. – Fourier transform of a sinusoid is a delta function – Delta function * any function = delta Function – Reference Introduction to Fourier Optics: Joseph Goodman 2-4 images are sufficient to map projector, (if range is small enough)

CONFIDENTIAL Project this:

CONFIDENTIAL Phase Reconstruction Reflectance, How much light Is coming from that location On the board If we have > 3 equations, now we can entertain least squares, or L1 error minimizing solutions. Modulation How much the Light levels are Changing as a function of phase shift Phase: What we are Searching for Phase Shifts: Known Observed Pixel Intensities

CONFIDENTIAL Put this all together:

CONFIDENTIAL Put this all together: Roughly 3 Gigabits/second

CONFIDENTIAL Where to go from here: Shadows: Target Receiver Light Projector No Data here

CONFIDENTIAL Where to go from here: Shadows: Target Receiver Light Projector Now we have double data on top, how to combine?

CONFIDENTIAL What else are we concerned about? Our customers aren’t interested in a “height map” They are interested in measuring relative heights in the field of regard. They are interested in summary measures. – Volume of target – Average height of target above FR4 – Max height of target above FR4 This is very similar to standard image processing techniques – Segmentation – Image smoothing / Noise Reduction – References: Geometric Partial Differential Equations and Image Analysis : Guillermo Sapiro Digital Image Processing : Gonzalez and Woods Computer Vision: Algorithms and Applications; Richard Szeliski

CONFIDENTIAL So what skill set does one need? Mathematical Modelling Understanding of software, and algorithm complexity Statistics Fourier Analysis Estimation Theory Machine Vision techniques Machine Learning techniques Whatever the job requires….

CONFIDENTIAL Questions?