1. David Teitlebaum - Program Manager Jordan Parker - Developer Kurt Berglund - DeveloperKurt Berglund - Developer Microsoft CorporationMicrosoft Corporation

2. Introduction to WPF3D What do you need to do to see something? Interactive 2D on 3D Make VisualBrush work like you expected it to Performance Tips Learn how to avoid common mistakes

4. Pattern: FooBar is a Bar with a Foo Examples: ModelVisual3D is a Visual3D with a Model3D GeometryModel3D is a Model3D with a Geometry3D MeshGeometry3D is a Geometry3D with a mesh ScaleTransform3D is a Transform3D with a scale And so on (generally) DrawingImage vs. ImageDrawing anyone?

5. Viewport3D Visual3D Camera Model3DGroup GeometryModel3D Light Material Geometry3D Let’s do a simple 3D video example step-by-step

6. Right-handed Notice that +Y is up… opposite of 2D! (Image from http://www.willamette.edu/~gorr/)

7. Specifies a mesh of triangles Positions: Triangle vertices TriangleIndices: Optional indices Normals: Per-vertex orientation used in lighting. Generated for you. TextureCoordinates: Describes the 2D Brush mapping. NOT generated for you. Not needed for SolidColorBrushes. Triangle vertices should be given in CCW order Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

8. If specified: positions[indices[0]] positions[indices[1]]… texcoords[indices[0]] texcoords[indices[1]]… normals[indices[0]] normals[indices[1]]… If not specified: positions[0] positions[1]… texcoords[0] texcoords[1]… normals[0] normals[1]… Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

9. (image from blogs.msdn.com/danlehen) Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

10. If you don’t specify normals, we generate them by averaging the “real” normals of the triangles that share the vertex. This gives a smooth appearance If you duplicate a vertex, it will end up with different normals each time. Triangle specification matters! If you specify too many, we throw out the extras If you specify too few, we generate the rest Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

11. All Vertices SharedEdge Vertices Duplicated Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

12. (-1,1,0) 0 (0,0) (-1,-1,0) 1 (0,1) (1,-1,0) 2 (1,1) (1,1,0) 3 (1,0) Key: pos idx tc Looking down -Z Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

13. <MeshGeometry3D Positions=“-1,1,0 -1,-1,0 1,-1,0 1,1,0” TextureCoordinates=“0,0 0,1 1,1 1,0” TriangleIndices=“0,1,2 2,3,0” /> We’ll let WPF compute the normals In general, you’ll be exporting a mesh to XAML from a tool Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

14. Reflection/absorption of light EmissiveMaterial: Surface appears to give off light SpecularMaterial: Shine DiffuseMaterial: Soft scattering Takes a Brush for the base color Combine them with MaterialGroup When in doubt, use DiffuseMaterial Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

15. DiffuseEmissiveSpecular Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

16. (In code, you could use a DrawingBrush w/ a VideoDrawing instead) Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

17. Combines the Material and the Geometry3D Allows you to specify a transform (the plane mesh) (the video material) Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

18. Not unlike reality, without light you can’t see anything* AmbientLight: Equal contribution everywhere DirectionalLight: The sun PointLight: Light bulb SpotLight: Self explanatory Lights are Model3Ds and thus have transforms We’ll use an AmbientLight in our example for simplicity Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D * Except EmissiveMaterials

19. (the ambient light) (the plane GeometryModel3D) Group multiple Model3Ds together Is a Model3D Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

20. 3D analog of the 2D Visual class You need at least one just like you need at least one Visual in 2D (FrameworkElements are Visuals) To draw a Model3D, use ModelVisual3D (the Model3DGroup) Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

21. Determine what part of the scene you see and project 3D onto a 2D plane OrthographicCamera: parallel lines stay parallel PerspectiveCamera: perspective foreshortening MatrixCamera: Specify your own projection <PerspectiveCamera Position=“0,0,4” LookDirection=“0,0,-1” /> Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

22. The bridge between the 2D and 3D world It is a FrameworkElement Takes a Camera and the root of your Visual3D tree (the PerspectiveCamera) (the ModelVisual3D) Viewport3D PerspectiveCamera AmbientLight DiffuseMaterial ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D

26. Interactive3DDecorator Goes around the Viewport3D you want to make interactive InteractiveVisual3D Subclass of ModelVisual3D used to add interactive 2D on 3D Has DependencyProperties Geometry, Visual, and Material used to set the appearance of the object

27. <VisualBrush Visual=“{StaticResource Visual1}” />

28. <local:InteractiveVisual3D Geometry=“{StaticResource PlaneMesh}” Visual=“{StaticResource Visual1}” />

29. Hidden visual layer placed above Viewport3D 2D on 3D that the mouse is over is placed in hidden layer Layer moved so that the point the mouse is over in 3D is the same as the point the mouse is over in the hidden layer

30. public void HitTest(object sender, System.Windows.Input.MouseButtonEventArgs args) { Point mouseposition = args.GetPosition(myViewport); PointHitTestParameters pointparams = new PointHitTestParameters(mouseposition); VisualTreeHelper.HitTest(myViewport, null, HTResult, pointparams); } public HitTestResultBehavior HTResult(System.Windows.Media.HitTestResult rawresult) { RayHitTestResult rayResult = rawresult as RayHitTestResult; if (rayResult != null) { RayMeshGeometry3DHitTestResult rayMeshResult = rayResult as RayMeshGeometry3DHitTestResult; if (rayMeshResult != null) { … } return HitTestResultBehavior.Continue; }

31. RayMeshGeometry3DHitTestResult contains all the information about the hit test Point, Mesh, Model, Visual3D hit Vertex indices of triangle hit Barycentric coordinates for point of intersection: can be used to compute texture coordinate of intersection

32. Weighted sum of triangle’s vertices p = VertexWeight1 * v1 + VertexWeight2 * v2 + VertexWeight3 * v3 Can be extended to texture coordinates uv = VertexWeight1 * uv1 + VertexWeight2 * uv2 + VertexWeight3 * uv3 Use to figure out where on texture you are

33. Say we want to fill the viewport with a 3D rectangle Viewport3D has width, height, and camera is at origin looking down negative z axis Horizontal FieldOfView -z

34. Can bind any DependencyProperty <ScaleTransform3D ScaleX="{Binding ElementName=sliderX, Path=Value}" ScaleY="1" ScaleZ="1" />

36. Performance must be a priority throughout app construction Test early, test often Do not assume you can easily fix performance problems after your application is functionally complete Test on the hardware you care about Old Desktops, New Desktops, Laptops Most Dev machines are not “Old Desktops”

37. CPU Easy to measure with task manager, profilers, etc. Many potential causes of CPU load GPU Hard to measure – there are few public tools to measure GPU load Can degrade frame rates without taxing CPU Few potential causes – typically the result of insufficient fill rate on the GPU or background video decode GPUs are optimized for video games with few state changes

38. Coarse indicator of maximum machine capabilities Queryable via: (RenderCapability.Tier >> 16) Tier resides in the high word so don’t forget the shift! Indicates static capabilities An upper bound on what can be expected Does not account for dynamically consumed machine resources Value is read only (unfortunately! )

39. Rendering TierShader VersionVideo RAMMultitexture Units 0NA 11.030 MB2 22.0120 MB4

40. Performance tools WPFPerf.exe Profilers Microsoft F1 AMD Code Analyst Intel Vtune

42. #1 Brush Selection Persistence vs. Nonpersistence of realizations Intermediate Render Targets Texture memory swizzling Texture size Smaller is better Watch out for automatic texture sizing RenderTargetBitmap - best of both worlds?

43. #1 Continued - RenderTargetBitmap Rectangle r = new Rectangle(); r.Width = 50; r.Height = 50; r.Fill = new LinearGradientBrush(Colors.CornflowerBlue, Colors.PeachPuff, 13.37); r.Measure(new Size(50, 50)); r.Arrange(new Rect(new Size(50, 50))); r.UpdateLayout(); RenderTargetBitmap rtb = new RenderTargetBitmap((int)r.ActualWidth, (int)r.ActualHeight, 96.0, 96.0, PixelFormats.Pbgra32); rtb.Render(r);

44. #1 Continued - Brush Speed on Tier-2 Hardware in 3D ONLY Brush Speed (Fastest to Slowest)PersistenceIRT 1SolidColorBrushYesNo 2LinearGradientBrushYesNo 3ImageBrushYes“No” 4DrawingBrush (cached)Yes 5VisualBrush (cached)Yes 6RadialGradientBrushNoYes 7DrawingBrush (uncached)NoYes 8VisualBrush (uncached)NoYes

45. #2 Disable Viewport3D.ClipToBounds WPF uses complex antialised clipping When Viewport3D.ClipToBounds == False, 3D content can exceed 2D region of Viewport3D Implementation of clipping is slow enough that it should be disabled when it is not needed

46. #3 Disable Viewport3D.IsHitTestVisible WPF3D performs mouse-over hit testing on the CPU Mouse cursor is projected into object space and intersected with GeometryModel3D’s axis-aligned bounding box If bounding box is intersected, ray of projected mouse cursor must be intersection-tested against every triangle in GeometryModel3D, all on the CPU If you don’t need hit testing, disable it.

47. #4 GeometryModel3D Coalescence Each Model3D requires a change of GPU state Each requires the projection of its axis-aligned bounding box to device coordinates A Few complex GeometryModel3Ds is vastly preferable to many simple GeometryModel3Ds

49. #5 Minimize Mesh Animations They do not scale well due to platform architecture Entire Collections are remarshalled rather than collection deltas Disconnect meshes from 3D scene before modifying to minimize change notifications Keep animated meshes small (a few thousand vertices at the most)

50. #6 Disable 3D Antialiasing On Vista, WPF3D will try to render with 4 samples/pixel multisampling On XP, WPF3D will not try to multisample To prevent WPF3D from attempting to render with multisampling on Vista, add the attached property RenderOptions.EdgeMode=“Aliased” to your Viewport3D

51. #7 Text in 3D Text is expensive Glyphs are abstractly defined by Beziers Glyphs are generated and cached according to scale Small glyphs – Bitmaps / Large glyphs - Polygons scale changes cause glyph cache invalidation, glyph regeneration, and glyph cache repopulation RenderTargetBitmap / ImageBrush is strongly recommended for static text

52. #8 Caching of VisualBrush and DrawingBrush Attached Property - RenderOptions.CachingHint Use when ImageBrush is not sufficient Still slower than ImageBrush due to less aggressive texture memory swizzling on D3D’s RenderTargetTextures Frequency of cache regeneration can be controlled by RenderOptions.CacheInvalidationThresholdMinimum and RenderOptions.CacheInvalidationThresholdMaximum

53. #9 BitmapEffects Should only be used on static 3D content Implemented on the CPU Forces affected content to also be rendered on the CPU Great for static content, otherwise don’t use them

55. WPF3D http://blogs.msdn.com/wpf3d http://blogs.msdn.com/danlehen http://blogs.msdn.com/wpfsdk/archive/2007/01/15/maximizing-wpf- 3d-performance-on-tier-2-hardware.aspx http://www.codeplex.com/3dtools WPF Perf http://blogs.msdn.com/timothyc http://blogs.msdn.com/kiranku Ask Questions http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=119 news://microsoft.public.windows.developer.winfx.avalon Books Windows Presentation Foundation Unleashed Petzold’s book coming out Summer 2007

56. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.