1. A UGMENTED F UTURE OF M OBILE R EALITY Oleg Novosad Mobile / Web / Game Engineer

2. Agenda What is Augmented Reality? Basic augmented reality concepts It is everywhere Show me some code! Targets in augmented reality Tools for rich UX Bigger example (Vuforia Intro) Q&A

3. Virtual objects + Reality = Augmented Reality “AR augments the real world environment with virtual information improving people’s senses and skills.” - Ronald Azuma, A Survey of Augmented Reality

4. 1960’s - Boeing - hands-free flight navigation for pilots 1966 - Prof. Ivan Sutherland - Head Mounted Display (HMD) 1992 - LB Rosenberg - Air Force - Virtual Fixtures KARMA - Knowledge-based Augmented Reality for Maintenance Assistance 1999 - Nara Institute of Science and Technology – First AR Toolkit 2000 - AR Quake - First AR Video Game

5. Basic augmented reality concepts

6. Reality Source of reality Canvas Virtuality Native elements OpenGL (± OpenCV) Tools Application Target Idea

7. It is everywhere

9. “Annual revenues generated by mobile AR applications and services are expected to approach $1.5 billion by 2015 and more than $10 billion by 2020, up from less than $2 million in 2010.” – Juniper Research

10. Code examples: case 1 – “virtual buttons”

11. Code examples: case 2 – “OpenGL” public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Now let's create an OpenGL surface. glView = new GLSurfaceView(this); // To see the camera preview, the OpenGL surface has to be created translucently. glView.setEGLConfigChooser(8, 8, 8, 8, 16, 0); glView.getHolder().setFormat(PixelFormat.TRANSLUCENT); // The renderer will be implemented in a separate class, GLView. glView.setRenderer(new GLClearRenderer()); // Now set this as the main view. setContentView(glView); // Now also create a view which contains the camera preview... cameraView = new CameraView(this); //...and add it, wrapping the full screen size. addContentView(cameraView, new LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT)); } Set OpenGL as Content View, add camera view (actual surface view) to content Configure Renderer public class GLClearRenderer implements Renderer { public void onDrawFrame(GL10 gl) { float c = 1.0f / 256 * (System.currentTimeMillis() % 256); gl.glClearColor(c, c, c, 0.5f); gl.glClear(GL10.GL_COLOR_BUFFER_BIT); } public void onSurfaceChanged( GL10 gl, int width, int height ) { // This is called whenever the dimensions of the surface have changed. // We need to adapt this change for the GL viewport. gl.glViewport(0, 0, width, height); } public void onSurfaceCreated(GL10 gl, EGLConfig config) { // No need to do anything here. }

12. Extend SurfaceView to handle camera public class CameraView extends SurfaceView implements Callback { private Camera camera; public CameraView(Context context) { super(context); // We're implementing the Callback interface and want to get notified // about certain surface events. getHolder().addCallback(this); // We're changing the surface to a PUSH surface, meaning we're receiving // all buffer data from another component - the camera, in this case. getHolder().setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS); } public void surfaceCreated(SurfaceHolder holder) { // Once the surface is created, simply open a handle to the camera hardware. camera = Camera.open(); } public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) { // This method is called when the surface changes, e.g. when it's size is set. // We use the opportunity to initialize the camera preview display dimensions. Camera.Parameters p = camera.getParameters(); p.setPreviewSize(width, height); camera.setParameters(p); // We also assign the preview display to this surface... try { camera.setPreviewDisplay(holder); } catch(IOException e) { e.printStackTrace(); } //...and start previewing. From now on, the camera keeps pushing preview // images to the surface. camera.startPreview(); } public void surfaceDestroyed( SurfaceHolder holder ) { camera.stopPreview(); camera.release(); camera = null; }

13. T ARGET IN A UGMENTED R EALITY

14. Target – is your canvas Location Markers – Frame marker – QR-code and barcode – Custom marker Text Images – Predefined – User-defined Objects – Simple shapes – 3D objects – Complex geometries – People

15. Quality Features ARPA ARLab and other ARPA ARLab and other Layar D’Fuion Layar D’Fuion Tools for rich UX Cortexia DroidAR Cortexia DroidAR Wikitude Vuforia

16. FYI Resources AR tools comparison http://socialcompare.com/en/comparison/augmented-reality-sdks http://socialcompare.com/en/comparison/augmented-reality-sdks Top iOS AR apps http://www.iphoneness.com/iphone-apps/best-augmented- reality-iphone-applications/ http://www.iphoneness.com/iphone-apps/best-augmented- reality-iphone-applications/ AR Future in Mobile http://www.digitaltrends.com/mobile/future-ar-mobile/ AR Future http://thenextweb.com/dd/2015/08/31/augmented-reality-is-the- future-of-design/ Google :)

17. Q UESTIONS ?

18. T HANK YOU !