1. Graphics with Canvas, SurfaceView, and multitouch processing (panning and multitouch zoom) www.eecis.udel.edu/~bohacek GraphicsWithCanvas_2012.pptx

2. Approaches for Graphics Load image from /res/drawable – Best for static images OpenGL ES – 3D graphics (i.e., transforms such as spin can be applied to graphical objects) – Best for game-type animation Draw on Canvas or SurfaceView – Canvas for drawing within the UI thread – SurfaceView is faster, and better for detailed graphics Note, if your main thread take too long, the OS will kill it, and it will be difficult to debug.

3. Drawable shapes Make new app, edu.udel.eleg454.Graphics1 In onCreate is setContentView(R.layout.main) Instead of the view generated by R.layout.main, we use our own, which extends View In Graphics1 class, add private class MyView extends View { public MyView(Context context) { super(context); } @Override protected void onDraw(Canvas canvas) { ShapeDrawable mDrawable = new ShapeDrawable(new OvalShape()); mDrawable.getPaint().setColor(0xff74AC23); mDrawable.setBounds(10, 10, 310, 60); mDrawable.draw(canvas); } Then, in onCreate, replace setContentView(R.layout.main); with setContentView(new MyView(this)); Run Besides OvalShape, ArcShape, PathShape, RoundRectShape, Shape, and BitMaps

4. View Widget The previous method required us to replace setContentView(R.layout.main); This resulted in the entire view being controlled by our view object – E.g., we could not have a button in the view where we place the button with the layout editor To fix this, we add a view widget Move MyView to separate class Make new class – In package explorer, under /src – Find edu.udel.eleg454.TestGraphics1 – Right click on edu.udel.eleg454.TestGraphics1 – Select: new->class – Dialog opens Name: MyView Superclass: View (then select browser to get full name: android.View) OK Move functions from private class MyView to this new MyView – Move public MyView( – Move onDraw Also, in MyView add public MyView(Context context, AttributeSet attrs) { super(context, attrs); }

5. View Widget Go to main.xml graphical layout editor – Drag button to the screen – Leave id as button1 Go to main.xml (not the editor) Find second Before, add – – Note that edu.udel.eleg454.TestGraphics1.MyView is the name of the separate class. If another name is used, then this name should be changed – Save and go back to graphical view. There should be a box labeled MyView. Drag the box to make it larger Run

6. Canvas Drawing Canvas has many drawing functions, e.g., drawPath(Path path, Paint paint) In onDraw, add the following Path: sequence of graphical objects – Path path = new Path(); – Make line between two points path.moveTo(10,10); // starting place path.lineTo(160,160); – add circle somewhere path.addCircle(160,160,20, Path.Direction.CCW); Paint – for setting color and line width – Paint paint = new Paint(); – paint.setDither(true); – paint.setColor(Color.RED); – paint.setStyle(Paint.Style.FILL_AND_STROKE); – paint.setStrokeJoin(Paint.Join.ROUND); – paint.setStrokeCap(Paint.Cap.ROUND); – paint.setStrokeWidth(10); Draw view canvas.drawPath(path, paint); run

7. Change graphics At the end of TestGraphics1Activity.onCreate MyView myView = (MyView) findViewById(R.id.View01); Button button = (Button)findViewById(R.id.button1); button.setOnClickListener(new View.OnClickListener() {}); – Let eclipse add unimplemented methods In onClick, add – myView.redraw(); In MyView – Add class variable int radius = 20; – In onDraw, change path.addCircle(160,160, 20, Path.Direction.CCW); To path.addCircle(160,160,radius, Path.Direction.CCW); – Add function public void redraw() { radius = 80; invalidate(); // this is needed to force redraw } run

8. notes Use invalidate() to force redraw – In Graphics1, add variable MyView myView; – In Graphics1.onCreate(), add myView = (MyView) findViewById(R.id.View01); – Then myView.invalidate(); will force redraw Is canvas documentation for more graphics – E.g., drawBitmap has several functions Avoid declaring and setting variables in onDraw, instead, setting them elsewhere and access them from draw Use invalidate() to force redraw Use SurfaceView for faster screen drawing

9. SurfaceView Faster You can draw on a SurfaceView from other threads, not just to UI thread – When drawing with the UI thread, if the drawing takes a long time, then everything else must wait for the drawing to complete, e.g., the user cannot press any buttons If you put a long activity in the UI thread, a message will pop up saying that the app has stopped responding If you put a long drawing activity when starting the app, the system just kill it (thinking that it did not start correctly) But, SurfaceViews are not transparent, nothing behind the view can be seen Differences – In Canvas approach, your onDraw function is called and has argument canvas. You can draw on this canvas. You can force a redraw can calling invalidate. Invalidate will result in onDraw being called from the UI thread – With surfaceview, you get a canvas and can draw on it whenever you want. Usually you draw on it from a new thread E.g., You start the thread from the UI thread

10. SurfaceViewFun Make a new app, SurfaceViewFun and package name edu.udel.eleg454.SurfaveViewFun Make new class – Right click on edu.udel.eleg454.SurafceViewFun – Select New -> class – Name: MySurfaceView – SuperClass: SurfaceView Go to res/layout/main.xml Open graphical view Click on “Advanced” Drag SurfaceView In xml view, find the <SurfaceView …. Change <SurfaceView … to <edu.udel.eleg454.SurfaceViewFun.MyView Go back to graphical view and check that the surfaceView is now labeled MySurfaceView. If not, then something is wrong.

11. MySurfaceView Open MySurfaceView Eclipse will ask to add some unimplemented functions. Add all three of these – public MySurfaceView(Context context, AttributeSet attrs, int defStyle) – public MySurfaceView(Context context, AttributeSet attrs) { – public MySurfaceView(Context context) { Each on has content – super(context, attrs); // eclipse migth add this part – ini(context); // a function to make

12. Change – public class MySurfaceView extends SurfaceView To – public class MySurfaceView extends SurfaceView implements SurfaceHolder.Callback – Add unimplemented functions Add member variables – SurfaceHolder surfaceHolder; // needed fro drawing – MyThread myThread; // will make MyThread next

13. Thread class The whole reason to use a surfaceView is to draw on a different thread then the UI thread. Let’s make a thread class. Make new subclass in MySurfaceView and extend Thread, i..e, add – class MyThread extends Thread {}; This will be the thread we use for drawing We will draw an oval, by the drawing is slightly animated In MyThread, add member variables – SurfaceHolder surfaceHolder = null; // this a key variable as it allow use to get a canvas – boolean done; – long startTime; – Canvas canvas; – RectF rectForOval = null; – float duration = 5*1000; – Paint drawingPaint; – float arcSweep;

14. MyThread member functions Constructor – public MyThread(SurfaceHolder _surfaceHolder) { surfaceHolder = _surfaceHolder; drawingPaint = new Paint(); drawingPaint.setColor(Color.BLUE); – } We will draw an oval, but it will be slightly animated This is the function that will run in the thread. – @Override – public void run() { Log.e("surface","running thread"); // some initialization setOval(); startTime = System.currentTimeMillis(); arcSweep = 0; done=false; while (!done) { // draw until done – updateArcSweep(); – drawCurrentArc(); } Log.e("SurfaceViewFun","MyThread.run has finished"); – }

15. Add the following functions to MyThread public void setOval() { – rectForOval = new RectF(10,10,300,600); } void updateArcSweep() { – long currentTime = System.currentTimeMillis(); – arcSweep = (float) ((currentTime-startTime)/duration*360.0); – if (currentTime-startTime>duration) { done = true; arcSweep = 360; – } } void drawCurrentArc() { – canvas = surfaceHolder.lockCanvas(null); // must lock before drawing – canvas.drawColor(Color.BLACK);// clear the screen – canvas.drawArc(rectForOval, 0, arcSweep, true, drawingPaint); // draw new stuff – surfaceHolder.unlockCanvasAndPost(canvas); // must unlock when done }

16. MySurfaceView Initialize (this function is called by each of the MySurfaceView constructors – public void ini(Context context) { Log.e("SurfaceViewFun","ini"); surfaceHolder = getHolder(); // MySurfaceView extends SurfaceView, which has member function getHolder. We use the surfaceHolder to get the canvas that we will draw on surfaceHolder.addCallback(this); // MySurfaceView implements SurfaceHolder.CallBack. This allow MySurfaceView to get message about when the surface is ready for drawing and whether the surface has change (e.g., change orientation) myThread = new MyThread(surfaceHolder); // make thread setFocusable(true); – } In surfaceChanged, add – Log.e("SurfaceViewFun","surface changed. Width: "+width+" height: "+height); Start thread – Since we extend SurfaceHolder.Callback, we implement surfaceCreated. – When this function is called, the surface is ready for drawing (the surface might not be ready for drawing when the SurfaceView constructor is called. We need to wait for the surfaceCreated function) – In surfaceCreated, add Log.e("SurfaceViewFun","created"); myThread.start(); // starts thread. Will lead to MyThread.run being called in its own thread Run it

17. playing Compare directly writing onto canvas vs using a surfaceview and a thread In surfaceCreated,change – myThread.start(); – To – myThread.run(); Change – class MyThread extends Thread { To – class MyThread { Change – @Override – public void run() { … To – //@Override – public void run() { …. – ]that is, comment out the override statement So now there is no thread, we writing from the UI thread. Run it Undo what we did so it draws in a thread

18. Multi-touch and zoom View objects process touches. We need to implement this processing to capture multi-touch. Also, to process some touches, we need GestureDetectors to help In summary – Override onTouchEvent – Extend ScaleGestureDetector.SimpleOnScaleGestureListener – Extend GestureDetector.SimpleOnGestureListener There is a bunch of code, but only a couple of critical spots In MySurfaceView::ini, add – iniTouchHandling(context); At the end of MySurfaceView, add – private float mPosX =0, mPosY = 0; // will indicate how much we have panned. Use these to adjust the graphics – float mScaleFactor = 1.f; // indicate the scalling. Use this to adjust graphics – private float mLastTouchX; – private float mLastTouchY; – private static final int INVALID_POINTER_ID = -1; – private int mActivePointerId = INVALID_POINTER_ID; – GestureDetector mTapListener; – ScaleGestureDetector mScaleDetector; – public void iniTouchHandling(Context context) { mScaleDetector = new ScaleGestureDetector(context, new ScaleListener()); mTapListener = new GestureDetector (context, new TapListener()); – }

19. onTouchEvent @Override public boolean onTouchEvent(MotionEvent ev) { – // let our gesture detectors process the events – mScaleDetector.onTouchEvent(ev); – mTapListener.onTouchEvent(ev); – final int action = ev.getAction(); – switch (action) { – case MotionEvent.ACTION_DOWN: { final float x = ev.getX(); final float y = ev.getY(); // Remember where we started mLastTouchX = x; mLastTouchY = y; mActivePointerId = ev.getPointerId(0); break; } – //More on next slide

20. onTouchEvent (continued) case MotionEvent.ACTION_MOVE: { – final int pointerIndex = ev.findPointerIndex(mActivePointerId); – final float x = ev.getX(pointerIndex); – final float y = ev.getY(pointerIndex); – if (!mScaleDetector.isInProgress()) { // Calculate the distance moved final float dx = x - mLastTouchX; final float dy = y - mLastTouchY; // Move the object mPosX += dx; mPosY += dy; // Remember this touch position for the next move event mLastTouchX = x; mLastTouchY = y; // Invalidate to request a redraw // invalidate(); // use this is a regular canvas is being used myThread.setOval(); if (myThread.done==true) – myThread.drawCurrentArc(); – } – break; } The current amount that we have panned With new values of mPosX and mPosY, we need to update the graphics -We can remake the oval -If we are still drawing, the the new oval will be drawn -If we have finished drawing, tehn we need to redraw -If we are not using a surface view (i.e., we are directly drawing on the canvas, then in order to have the new values of mPosX and mPosY be shown, we need to call invaliate for the view

21. onTouchEvent (end) case MotionEvent.ACTION_UP: { – mActivePointerId = INVALID_POINTER_ID; – break; } case MotionEvent.ACTION_CANCEL: { – mActivePointerId = INVALID_POINTER_ID; – break; } case MotionEvent.ACTION_POINTER_UP: { – // Extract the index of the pointer that left the touch sensor – final int pointerIndex = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) – >> MotionEvent.ACTION_POINTER_INDEX_SHIFT; – final int pointerId = ev.getPointerId(pointerIndex); – if (pointerId == mActivePointerId) { // This was our active pointer going up. Choose a new // active pointer and adjust accordingly. final int newPointerIndex = pointerIndex == 0 ? 1 : 0; mLastTouchX = ev.getX(newPointerIndex); mLastTouchY = ev.getY(newPointerIndex); mActivePointerId = ev.getPointerId(newPointerIndex); – } – break; } } // and switch statement return true; } // ends onTouch

22. ScaleGestureDetector.SimpleOnScaleGestureListener private class ScaleListener extends ScaleGestureDetector.SimpleOnScaleGestureListener { @Override – public boolean onScale(ScaleGestureDetector detector) { mScaleFactor *= detector.getScaleFactor(); // Don't let the object get too small or too large. mScaleFactor = Math.max(0.1f, Math.min(mScaleFactor, 5.0f)); //invalidate(); // use this for a regular canvas (i.e., not a SurfaceView) myThread.setOval(); if (myThread.done==true) myThread.drawCurrentArc(); return true; – } } Since mScaleFactor has changed, the graphics should be updated Also, if we are drawing directly on the canvas, then we need to invalidate sot nat onDraw is called

23. GestureDetector.SimpleOnGestureListener private class TapListener extends GestureDetector.SimpleOnGestureListener { @Override public boolean onDoubleTap(MotionEvent e) { – Log.e("SurfaceViewFun","double tap "+e.getX()+" "+e.getY()); – return true; } @Override public void onLongPress(MotionEvent e) { – Log.e("SurfaceViewFun","got long press at location x="+e.getX()+" y="+e.getY()); } @Override public boolean onFling(MotionEvent e1, MotionEvent e2, float velocityX, float velocityY) { – Log.e("SurfaceViewFun","fling: started at ("+e1.getX()+","+e1.getY()+"). Ended at ("+e2.getX()+","+e2.getY()+"). With velocity ("+velocityX+","+velocityY+")"); – return true; } @Override public boolean onScroll (MotionEvent e1, MotionEvent e2, float distanceX, float distanceY) { – Log.e("SurfaceViewFun","scroll: started at ("+e1.getX()+","+e1.getY()+"). Ended at ("+e2.getX()+","+e2.getY()+"). With total distance ("+distanceX+","+distanceY+")"); – return true; } Our app does not use these touches. But if you want them, here they are

24. Make graphics use mPosX, mPosY, and mScaleFactor Need to change the oval public void setOval() { – synchronized(surfaceHolder) { // make sure that we are not drawing while updating the oval rectForOval = new RectF(10*mScaleFactor+mPosX,10*mScaleFactor+mPosY,300*mScaleFactor+mPosX,600* mScaleFactor+mPosY); – } } drawCurrentArc also need to be synchronized void drawCurrentArc() { – canvas = surfaceHolder.lockCanvas(null); – canvas.drawColor(Color.BLACK); – synchronized(surfaceHolder) { canvas.drawArc(rectForOval, 0, arcSweep, true, drawingPaint); – } – surfaceHolder.unlockCanvasAndPost(canvas); } Run mPosX, and mPosY translate the oval mScaleFactor scales the oval We should make sure that no other thread is reading RectF ad we are resetting it. We synchronized with surfaceHolder