© 2010 Altera Corporation—Public Using Altera FPGAs to Implement Wide Dynamic Range (WDR) Image Sensor Pipelines (ISP) and Video Analytics 2010 Technology Roadshow

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Agenda IP Camera Market Trends Implement WDR ISP using FPGA Video Analytics Market Trends Implement Video Analytics using FPGA Summary 2

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. IP Camera Market Trends 3

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Video Surveillance IP Pipeline Median filters Dynamic contrast Color correction De-mosaic Gamma correction WDR compression Fish eyes distortion correction Image stabilization Edge enhancement 3D noise reduction Timing generator Image stitch Bad pixel replacement De-interlace Color format conversion Sensor format conversion Auto exposure Auto focus Auto white balance Motion Control Video ISP Video Compression & Analytics Video Interfacing ISP Video Analytics Interfacing CCC / CMOS Sensor Video Capture HD H.264 Video Compression Motion detection People counting Forbidden area Loitering Stolen objects Object Tracking Color space conversion Video format conversion Video stream multiplexer Video frame buffers 3G SDI PCIe PCIex4 Video Displays Display / Store Scaler De-interlace Graphic Overlay OSD Video Format Conversion 4

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. IP Camera System “ASSP” Configuration DSP ASSP Flash Altasens Aptina Full WDR ISP NIOS-II DDR2 SDRAM 2x12 bit Raw Data SPI 2A 1080P/720P 30/60fps S SPI YCrCb 4:2:2 Lens 5

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Copyright Apical 2009 Pixel-by-pixel Tone Curve Correction iridix automatically generates a different tone curve for every pixel of every frame    6

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. New Performance Level for HD WDR 2008 State of the Art NTSC resolution WDR up to 90dB Low image quality Custom chipset High cost HD 1080p resolution WDR up to 120dB Broadcast image quality Flexible FPGA chipset Competitive cost 2010 State of the Art 7

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Evaluation System Allows 720p60 & 1080p60 video from the Aptina WDR & Altasens sensor to be displayed on an external monitor HSMC Existing 3C120 Kit + Bitec DVI Card Existing Aptina / Altasens “Headboard” DVI PHY HSMC 8

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. EP3C120 Evaluation Board with Aptina M033 / Altasens 3372 WDR Sensor 9

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Advantages of Using FPGAs for ISP Availability—no current ASSP can support the latest HD WDR sensors Flexibility—different sensors have different interfaces Fast time to market—latest and best image quality High performance—full HD 1080/60p + WDR 120dB processing on Cyclone IV FPGAs Embedded CPU core—removes the need for an additional processor Competitive pricing—the total cost of ownership is low, and the ROI is high 10

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. WDR ISP Summary FPGAs offer unique image processing algorithms  Wide dynamic range (WDR) image sensor control and dynamic range compression can only be done with FPGAs for live HD video  Real-time, powerful video analytics are offered only through FPGAs  Image processing techniques such as 30+ fps fisheye correction and image stitching are not feasible on DSP devices FPGAs enable the most flexible interfaces for video  For a camera application, the FPGA can connect to any image sensor  Output video can be transmitted using a variety of digital standards (such as SDI), or packetized without external components  FPGAs are capable of providing multimedia over IP MAC layers (GigE Vision, Ethernet AVB, IEEE-1588, and so on…) Altera provides complete intellectual property for the security camera market 11

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Video Analytics Overview 12

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Video Analytics Market Trends Market Penetration 2% upto 40% by 2012 Mid/High-end account for ~50% by 2012 Source: IMS Research 2008 Embedded exceed PC-based by 2008 Market exceed $800m by 2012 VCA – Video Content Analysis 13

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Video Analytics Camera System Video Analytics Video Analytics Image Sensor Image Signal Processing H264 Compression Lens HD/SD CCD /CMOS Compressed Video Out ASSP DSP 14

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Video Analytics Manned monitoring of all video surveillance streams is impractical and ineffective Video Analytics provides automated, real-time monitoring of video streams  Real-time, continuous, monitoring  Frees security personnel for other tasks Video Analytics system can automatically take action based on events:  Trigger real-time alerts to notify safety and security personnel  Regulate transmission or compression of video – save valuable network bandwidth and storage space 15

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Video Analytics Multiple Rules Intelligent Motion Detection  Detects motion within a specific area of the video (trespassing, etc) Car License Plate Finding  Detect automobile license plates Object Recognition and Classification  Monitor for missing or unwanted objects (theft, abandoned object) Loitering  Determine the behavior of people Forbidden Area  Detect and determine location of pedestrians  Highlight possible threats to driver of vehicle Behavior Detection  Retail anti fraud and theft Remote Monitoring 16

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Key Challenges for Video Analytics Sequential processing architecture of DSP solutions have limited processing power impacting:  Ability to handle High Definition / Increased fps  Accuracy, quality and robustness of analytics  Capacity to run multiple rules simultaneously Cost of current products Ease of deployment and management Need to improved accuracy, adaptive capabilities for environments  Camera vibration - windy, passing vehicle  Luminance change - shadow, light change  Climate change – sunny, snow or rain  Background change – leaves swing, water ripple Intelligent, easy to use GUI for setup and management 17

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. MVE TM Advantages / Benefits FPGA solution  Single chip - reduced part count - lower cost  Smaller packaging  Design flexibility-evolve over time Configurable resolution / speed Configurable events for different application areas  People counting, object detection, forbidden zone, loitering, movement in wrong direction, others Multiple algorithms at the same time  A few or all! Reduced customer systems expense  Optimize bandwidth Flexible GUI for VMS integration Easy enhancement and upgrade Eutecus’ Approach 18

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. HD Video Content Analytics – Eutecus, Inc. MVE TM : Single-chip, lHD video analytics solution within low-cost FPGAs (65 nm nodes) Key MVE TM features (available from Q1 2010): -Up to 1MPix (HD720p) resolution content analysis at video frame-rate -Performance: all the rules all the time -Scalable and reconfigurable from low-end SD to high-end HD video Future roadmap (available from Q1 2011): -MVE TM based video analytics + compression as a single-chip solution within next gen FPGAs ( nm nodes) Multi-Core Video Analytics Engine (MVE TM ) SoC Solution in FPGA Demo videos at MVE TM, MVE TM -F, MV TM E-C, MVE TM -FC, … HD IP Cameras Network Video Recorders Digital Video Recorders (Bi-i Ref Design ) Indoor/ outdoor surveillance (Live) Indoor/ outdoor surveillance (Live /Forensic) 19

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Eutecus Video Analytics Multi-Core Video Analytics Engine  High performance analytics (Human vision)  Derived from solutions for high-end defense Five times the processing performance of DSP solution  HD Resolution, 30fps  All algorithms operating simultaneously  Improved performance for shadows, reflection, glare, uneven lighting, changing backgrounds and complex motion DDR Memory Video In Flash Memory Communication Metadata,System Setup InstantVision Embedded Running C-MVA IP Cores 20

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Outdoor Surveillance – Demo (Illustration) “Traffic monitoring” Functions (event detection): –Detecting and counting moving objects –Classifying all vehicles –Monitoring all vehicles in the bus lane –Detecting vehicles changing lanes Advanced features: –Adaptation to the background including changes in the illumination and slowly moving shadows –Handling dynamic shadows –Filtering out fog, rain and snow Enabling technology: –Efficient space-time algorithms –Video analytics boosted by FPGA based cellular multi-core solutions 21

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Indoor Surveillance – Demo (Illustration) “Behavior monitoring” Functions (event detection): –Detecting, counting moving objects –Classifying all persons –Monitoring persons in forbidden areas –Detecting loitering –Detecting lost-stolen objects Advanced features: –Adaptation to the background including changes in the illumination and slowly moving shadows –Handling dynamic shadows –Handling dynamic reflections Enabling technology: –Efficient space-time algorithms –Video analytics boosted by FPGA based cellular multi-core solutions 22

© 2010 Altera Corporation—Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. Summary Main technical advantages of MVE based solutions: –HD resolution analysis at video frame-rate –Performance: all the rules all the time –FPGA based MVE solutions are scalable and reconfigurable from low-end SD to high-end HD video –MVE will be improved concurrently with providing high quality compression (MPEG4/H.264) in a single-chip solution (HD720p & HD1080p) Main weaknesses of current video analytics: –Typical analytics resolution is QVGA/VGA at or below video frame-rate –Performance: one or only a few rules at a time –DSP (e.g. DaVinci platform) based solutions do not offer rapid improvement for high-end HD video 23

© 2010 Altera Corporation—Public Thank You! For more information visit: