A UGMENTED F UTURE OF M OBILE R EALITY Oleg Novosad Mobile / Web / Game Engineer

Agenda What is Augmented Reality? Basic augmented reality concepts It is everywhere Show me some code! Targets in augmented reality Tools for rich UX Bigger example (Vuforia Intro) Q&A

Virtual objects + Reality = Augmented Reality “AR augments the real world environment with virtual information improving people’s senses and skills.” - Ronald Azuma, A Survey of Augmented Reality

1960’s - Boeing - hands-free flight navigation for pilots Prof. Ivan Sutherland - Head Mounted Display (HMD) LB Rosenberg - Air Force - Virtual Fixtures KARMA - Knowledge-based Augmented Reality for Maintenance Assistance Nara Institute of Science and Technology – First AR Toolkit AR Quake - First AR Video Game

Basic augmented reality concepts

Reality Source of reality Canvas Virtuality Native elements OpenGL (± OpenCV) Tools Application Target Idea

It is everywhere

“Annual revenues generated by mobile AR applications and services are expected to approach $1.5 billion by 2015 and more than $10 billion by 2020, up from less than $2 million in 2010.” – Juniper Research

Code examples: case 1 – “virtual buttons”

Code examples: case 2 – “OpenGL” public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Now let's create an OpenGL surface. glView = new GLSurfaceView(this); // To see the camera preview, the OpenGL surface has to be created translucently. glView.setEGLConfigChooser(8, 8, 8, 8, 16, 0); glView.getHolder().setFormat(PixelFormat.TRANSLUCENT); // The renderer will be implemented in a separate class, GLView. glView.setRenderer(new GLClearRenderer()); // Now set this as the main view. setContentView(glView); // Now also create a view which contains the camera preview... cameraView = new CameraView(this); //...and add it, wrapping the full screen size. addContentView(cameraView, new LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT)); } Set OpenGL as Content View, add camera view (actual surface view) to content Configure Renderer public class GLClearRenderer implements Renderer { public void onDrawFrame(GL10 gl) { float c = 1.0f / 256 * (System.currentTimeMillis() % 256); gl.glClearColor(c, c, c, 0.5f); gl.glClear(GL10.GL_COLOR_BUFFER_BIT); } public void onSurfaceChanged( GL10 gl, int width, int height ) { // This is called whenever the dimensions of the surface have changed. // We need to adapt this change for the GL viewport. gl.glViewport(0, 0, width, height); } public void onSurfaceCreated(GL10 gl, EGLConfig config) { // No need to do anything here. }

Extend SurfaceView to handle camera public class CameraView extends SurfaceView implements Callback { private Camera camera; public CameraView(Context context) { super(context); // We're implementing the Callback interface and want to get notified // about certain surface events. getHolder().addCallback(this); // We're changing the surface to a PUSH surface, meaning we're receiving // all buffer data from another component - the camera, in this case. getHolder().setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS); } public void surfaceCreated(SurfaceHolder holder) { // Once the surface is created, simply open a handle to the camera hardware. camera = Camera.open(); } public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) { // This method is called when the surface changes, e.g. when it's size is set. // We use the opportunity to initialize the camera preview display dimensions. Camera.Parameters p = camera.getParameters(); p.setPreviewSize(width, height); camera.setParameters(p); // We also assign the preview display to this surface... try { camera.setPreviewDisplay(holder); } catch(IOException e) { e.printStackTrace(); } //...and start previewing. From now on, the camera keeps pushing preview // images to the surface. camera.startPreview(); } public void surfaceDestroyed( SurfaceHolder holder ) { camera.stopPreview(); camera.release(); camera = null; }

T ARGET IN A UGMENTED R EALITY

Target – is your canvas Location Markers – Frame marker – QR-code and barcode – Custom marker Text Images – Predefined – User-defined Objects – Simple shapes – 3D objects – Complex geometries – People

Quality Features ARPA ARLab and other ARPA ARLab and other Layar D’Fuion Layar D’Fuion Tools for rich UX Cortexia DroidAR Cortexia DroidAR Wikitude Vuforia

FYI Resources AR tools comparison Top iOS AR apps reality-iphone-applications/ reality-iphone-applications/ AR Future in Mobile AR Future future-of-design/ Google :)

Q UESTIONS ?

T HANK YOU !