1. T1.1- Analysis of acceleration opportunities and virtualization requirements in industrial applications Bologna, April 2012 UNIBO

2. Android and accelerators? Android is the most widely used operating system for mobile devices –Linux-based –Open source Which applications running on android-based devices could benefit from HW acceleration (GPPA, HWPU)? –Smartphones have a camera and increasingly more computationally powerful  image processing –innovative and attractive apps leveraging their portability and ubiquity

3. Computer Vision Computer Vision is a branch of computer science that includes many techniques to extract, characterize, and interpret information in visual images Scientific and industrial communities are showing a growing interest in developing Computer Vision (CV) algorithms on embedded systems

4. Augmented Reality Augmented reality (AR) is a live view of a real-world environment with virtual objects superimposed upon (or composited with) the current scene –Semantic context –Real-time constraints Layar is an augmented reality browser for Android and iOS –It uses sensor data (camera, compass, GPS, and accelerometer) to identify user location and field of view –It shows geo-located POI organized in layers –As of September 2011, Layar had 2993 layers

5. AR Algorithms A primary issue of augmented reality application is image registration, that is the process to derive real world coordinates from images A first step for image registration is the detection of feature points using proper algorithms OpenCV is a C/C++ library that includes many CV algorithms, including feature detectors –Android build is available!!!

6. Feature extraction kernels Android – OpenCV API Reference (http://opencv.itseez.com/)http://opencv.itseez.com/ features2d – Feature detection and description SIFT – Scale Invariant Feature Transform [Yuan09] SURF – Speeded Up Robust Features [Bay06] FAST – Detects corners using the FAST algorithm [Rosten10] [Yuan09], Y. Yuan, C. Shi, “Object tracking using SIFT features and mean shift”, Computer Vision and Image Understanding, 2009 [Bay06] Bay, H., Tuytelaars, T., Van Gool, L. “SURF: Speeded Up Robust Features”, 9th European Conference on Computer Vision, 2006 [Rosten10] Rosten, E.; Porter, R.; Drummond, T.;, "Faster and Better: A Machine Learning Approach to Corner Detection," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.32, no.1, pp.105-119, Jan. 2010

7. Example: FAST Algorithm For each image point, FAST examines the 16 pixels on a circle with radius 3 and center p A feature is detected iff the intensities of at least n contiguous pixels are all above or all below the intensity of p by a threshold t Most feature detector algorithms are inherently parallel, as they verify some properties for each point in the current image

8. Embedded Platforms for Benchmarking LG Optimus 2xPandaboardDragonBoard CPU Frequency L1 Cache (I/D) L2 Cache Main Memory Dual-Core Cortex-A9 1 GHz, per core (32KB / 32KB) per core 1 MB Shared 1GB LPDDR2-667 Dual-Core Cortex-A9 1 GHz, per core (32KB / 32KB) per core 1 MB Shared 1GB LPDDR2-400 Dual-Core Scorpion 1.2 GHz, per core (32KB / 32KB) per core 512KB Shared 1GB LPDDR2-333 ISM Consumer smartphone Low-cost dev board Advanced dev board

9. Feature Detection on Embedded Platforms This figure shows the speed-up for a scalable version of FAST on three different platforms –Fine-Grained Data-Level Parallelization  The main computation loop divides the image in multiple horizontal bands  regular memory access pattern –The measured speed-up is very limited

10. Fine-Grained Data-Level Parallelization We tested the same version of FAST using a multi-core virtual platform –the experimental speed-up is closer to the ideal one when the number of threads becomes comparable with the number of cores The number of cores is limited (max 4 in current generation) –A viable solution to exploit scalability is the use of accelerators

11. Other applications QoS requirements Virtualization specification