1. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Sound in games Think about truly memorable games – They almost always have excellent background music and sound effects – Legend of Zelda, PacMan, Katamari Damacy, Little Big Planet, Radiant Silvergun – Music and artwork style combine to create an overall tone, or mood, for a game – Done well, this substantially enhances the overall gameplay experience

2. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Finding/Making Sounds Where can you find music to use in your game? – Reminder: there is this legal framework called Copyright Law – Creative Commons: use licenses that may allow free, non-commercial use http://creativecommons.org/ http://archive.org Sites with Creative Commons licensed music – New Grounds, Jamendo http://www.newgrounds.com/audio/ http://www.jamendo.com/en/creativecommons/ Look for “Attribution, Non-commercial” – “No Derivative Works” is OK, so long as you don’t modify If you use in your game, make sure you provide attribution – Put name of artist in your game (About page, splash screen, etc.) – Is polite to send them an email telling them about the use—will make them jazzed

3. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Finding/Making Sounds (cont’d) Find someone to create music for you – Music student at UCSC, for example – One team has already put up announcements for a music person It has never been cheaper to create high quality music – Instruments, microphones, mixing technology are all at historically low prices – Has led to a proliferation of music – Biggest problem: finding an audience – Games provide a good audience – Sales of many videogames larger than most music album sales – For many musicians, might have larger audience for video game soundtrack than for traditional album

4. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Finding/Making Sounds (cont’d) Use your voice! – Your voice is wonderfully adaptable and expressive Consider: – Record a raw voice clip – Bring into an editing software suite – Tweak/filter/alter until it suits your game – Can do much worse… Tools – Audacity http://audacity.sourceforge.net/ Free, open source sound recorder/editor – FL Studio (grown-up commercial version of Fruity Loops) http://flstudio.image-line.com/documents/what.html

5. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Playing Sounds in XNA Two ways Hard (but powerful) way – XACT audio tool Cross-platform audio creation tool – Many neat features – Edit volume, pitch, looping of sound clips – Can easily group together sound clips Easy (and 95% sufficient) way – Use Simplified Sound API – Can start, stop, and pause sound playing – Much, much easier to use

6. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Simple Sound API Two ways to play music – As a song Good for background music, or other long sounds – As a sound effect Good for short duration sounds

7. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides XNA Simple Sound API Supported music types: wav, wma, mp3 Add sound into project Contents folder – Audio files treated like other files in content pipeline – Copy sound file into project Contents folder – Right-click on Contents folder inside Visual Studio C# Express Add  Existing Item … select audio file you just copied in – Will now be visible inside Visual Studio – Need to double-check the Content Processor Sound Effect – XNA Framework – sound effects Song – XNA Framework - songs

8. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides XNA Song API Create a variable of type Song – Used to load songs via the content pipeline – Song mySong; Load sound file – mySong = Content.Load (@”{name of song file without extension}”) To play a sound, call Play() method on MediaPlayer object – MediaPlayer.Play(mySong); To pause/resume, call Pause()/Resume() on MediaPlayer object – MediaPlayer.Pause(); // no argument – MediaPlayer.Resume(); // no argument

9. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides XNA Sound Effect API Create a variable of type SoundEffect – Used to load sounds via the content pipeline – SoundEffect soundEffect; Load sound file – soundEffect = Content.Load (@”{name of sound file without extension}”) To play a sound, call Play() method on SoundEffect object – Returns a soundEffectInstance object – Can use this to stop, pause, and restart sound – SoundEffectInstance soundEffectInstance = soundEffect.Play();

10. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Demo of Song and Sound Effect API // Demo of use of Songs and Sound Effects inside XNA Caution: Treating a song as a sound effect can lead to very long compile times – Solution: keep sound effects short

11. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides 3D in XNA

12. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides 2D to 3D Many current games use 3D graphics – Much more complex than 2D graphics – This course will provide a basic overview of 3D graphics – CMPS 160, 161, 164 provide greater depth Ratchet and Clank

13. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides 3D Camera Analogy – 2D is like putting stickers on a page Place sticker (sprite) at x,y coordinate If a sticker is placed at 50,50, you see it – 3D is like recording a video with a camera What is recorded (shown on screen) is what camera sees Can have objects in a scene that aren’t visible – Can have 3D object at 50,50,50, but if camera is pointing in the opposite direction, won’t see it! Introduces rotation – Camera can potentially be rotated around all 3 axes – Objects can also be rotated around 3 axes – Affects what shows up on screen

14. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides 3D Coordinate System 3D graphics requires use of x,y,z coordinates So, which direction is positive z? – Is it back away from you, or towards you? – Either choice would work, need to pick one Right handed vs left handed coordinate systems – XNA uses right handed coordinate system Place hands, palms up – Point fingers in direction of positive X – Curl fingers in direction of positive Y – Thumb is pointing in direction of positive Z Right-handed coordinate system

15. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Camera Camera is comprised of two Matrix objects – View matrix holds information on Location of camera in world Camera direction Camera orientation – Projection matrix holds information on View angle Aspect ratio Near and far plane Location (x,y,z) Orientation Direction

16. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Matrix Structure XNA provides a Matrix structure – A 4x4 matrix, in row vector layout Row vector matrices view vectors as a row from left to right column vector matrices view vectors as a column from top to bottom – Built-in matrix operations +, -, *, /, == – Also, convenience matrices Identity, Up, Down, Left, Right – Large number of convenience methods Rotations, views into 3D world, determinants, invert

17. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Vector3 Structure Represents either: – An X, Y, Z coordinate, or, – Distances along X, Y, Z coordinates (e.g., a vector) Often a unit vector – all values between 0 and 1 – X, Y, Z properties (floats) Built-in operators – +, -, *, /, ==, != Convenience vectors – UnitX, UnitY, UnitZ, Up, Down, Left, Right Many convenience methods – Interpolations, rotations, distance, dot product, normalization (x,y,z) coordinate x y z (x,y,z) vector

18. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Creating an XNA Camera – View Matrix View matrix – Use CreateLookAt method of Matrix structure – Parameters (all Vector3) cameraPosition – location of camera (x,y,z) cameraTarget – coordinate of point where camera is looking cameraUpVector – vector indicating up position cameraPosition (x,y,z) cameraUpVector cameraTarget (x,y,z)

19. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Creating an XNA Camera – Projection Matrix Projection Matrix – Use CreatePerspectiveFieldOfView method – Parameters (all floats) fieldOfView – angle of camera view, in radians – Typically 45degrees – pi/2 radians aspectRatio – Typically width of screen divided by height of screen nearPlaneDistance – Distance from camera to near viewing plane – Objects between camera and near plane are not shown! farPlaneDistance – Distance from camera to far viewing plane – Objects beyond far plane are not shown! cameraPosition (x,y,z)

20. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Drawing Triangles All complex 3D shapes seen in games are composed of a series of triangles – A triangle has 3 points, one for each corner Points are more typically known as verticies Minimum number of points to unambiguously define a plane VertexPositionColor object – Represents the x,y,z location of a vertex – Also has a color for the vertex –VertexPositionColor v = new VertexPositionColor(new Vector3(0,1,0), Color.Blue); – Need 3 verticies to draw a triangle

21. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Vertex Declaration XNA requires you to tell the graphics device what kind of vertex data you will be using – Unclear why XNA can’t just figure this out, or handle multiple types seamlessly – Probably due to structure of DirectX API, or capabilities of graphics hardware – For now, treat as a must-do, black box – Put following in your main, derived from Game class –GraphicsDevice.VertextDeclaration = new VertexDeclaration(GrahpicsDevice, VertexPositionColor.VertexElements);

22. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Actually drawing the triangles In XNA, all 3D rendering is handled by a shader – Shaders defined using High Level Shader Language (HLSL) – Permits creation of wide range of visual effects – More on shaders in a few classes XNA provides a default shader – Called BasicEffect – Will use this for now BasicEffect is a type of effect – Effects contain a series of EffectPass – Each pass handles some aspect of putting things on screen

23. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Using Basic shader Five steps Create Shader Copy over camera information Iterate through EffectPasses Examine source code from example in Chapter 9 of XNA 3.0

24. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Using Basic shader Five steps: 1.Create Shader –BasicEffect effect = new BasicEffect(GraphicsDevice, null); 2.Copy over camera information –effect.View = camera.view; –effect.Projection = camera.projection; 3.Set world matrix –effect.World = … (more on this in a few slides) 4.Enable vertex capabilities (varies by Vertex type) –Effect.VertexColorEnabled = true; // for VertexPositionColor –Effect.Texture = myTexture; // for VertexPositionTexture Effect.TextureEnabled = true; 5.Iterate through EffectPasses – Call to DrawUserPrimitives inside EffectPass puts triangles on screen

25. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Iterating Through Effect Passes Each Effect has calls to begin()/end() Effects are comprised of passes – Each pass requires a call to begin()/end() effect.Begin(); foreach (EffectPass pass in effect.CurrentTechnique.Passes) { pass.Begin(); GraphicsDevice.DrawUserPrimitives (PrimitiveType.TriangleStrip, verts, 0, 1); pass.End(); } effect.End(); Draws verticies Passes # of triangles (the “primitive shape” in this context) to draw Index into verts array

26. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Triangle Drawing Examine this line of code –GraphicsDevice.DrawUserPrimitives (PrimitiveType.TriangleStrip, verts, 0, 1); – What is a TriangleStrip? Three ways to draw triangles – Triangle List Each set of three verticies defines a triangle Memory inefficient, since triangles often share edges in complex 3D meshes – Triangle Strip Builds triangle out of first three verticies Each additional vertex creates new triangle using that vertex, plus previous two verticies – Triangle Fan Each additional vertex creates new triable using that vertex, the previous vertex, plus the first vertex http://escience.anu.edu.au/lecture/cg/surfaceModeling/image/ surfaceModeling015.png

27. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides World Matrix Each triangle has 3 verticies, and each vertex has an x,y,z position – This position is given with respect to an origin location – That is, location is with respect to a local coordinate system World matrix – Translates from the local coordinate system to the world (i.e., visible on screen) coordinate system Local coordinate system offset, no rotation (Note: example uses left handed coordinate system, XNA uses right-handed coordinates) Source: MSDN DirectX documentation Local coordinate system offset and rotated www1.adept.com/main/KE/DATA/V%20Plus/V%20Language%20User/i mages/World+yaw.gif local world

28. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Translation A translation shifts the local coordinate system relative to the world coordinate system XNA provides a method for this – Matrix.CreateTranslation 3 parameters are x,y,z movements Matrix.CreateTranslation(0.01f, 0, 0); // Shift right (positive) along x axis Matrix.CreateTranslation(-0.01f, 0, 0); // Shift left (negative) along x axis Multiply world matrix by translation matrix to cause shift – All translations and rotations in 3D graphics accomplished via matrix multiplication

29. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Rotation A rotation shifts the local coordinate system by an angle relative to the world coordinate system XNA helper methods – Matrix.CreateRotationX, Matrix.CreateRotationY, Matrix.CreateRotationZ Rotations around single axes Matrix.CreateRotationY(angle in radians); – Matrix.CreateFromYawPitchRoll Rotate around multiple axes Matrix.CreateFromYawPitchRoll(yaw rad., pitch rad., roll rad.) Demonstration of example triangle drawing code

30. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Triangles are fine, but models are better Today’s 3D games have a large number of objects – Theoretically possible to create these objects by manually writing C# code to create each individual triangle – In practice, this is rarely done. – Far better to use a 3D modeling tool Maya, XSI, 3DS Max, Blender – Allows artists to create objects in world – Allows programmers to focus on behavior of objects – Modeling tools permit much faster creation and editing of models

31. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides 3D Modeling Several ways to model a 3D object – Polygonal modeling Objects are subdivided into a series of polygons (triangles) Can only approximate curved surfaces Dominant modeling form in computer games and computer graphics due to speed of rendering – NURBS Surfaces are defined by spline curves Curves defined and controlled by control points – Splines and patches Curved lines define surfaces. Between polygons and NURBS – Primitives modeling Objects built up from primitive shapes (balls, cubes, cylinders, etc.)

32. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides 3D Model formats There is a huge number of 3D model formats – No dominant standard – Interchange among models is often lossy XNA supports two 3D model formats –.X (DirectX) –.FBX Originally for FilmBox by Kaydara, then Alias, now Autodesk

33. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Models in XNA Model – Represents some entity A person, a car, or potentially a complex scene with many parts – Typically used to represent one logical object (e.g., a person) that has multiple parts (head, arms, legs, etc.) – A model contains multiple meshes and bones Mesh – Represents a single physical object – Triangles, textures, shaders, etc. – XNA ModelMesh class Bone – Represents placement of each mesh relative to other meshes – A transformation matrix Model ModelMesh 1 N Car Body +bone Wheel +bone Door +bone Bone 1 N

34. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Models in XNA (cont’d) ModelMesh contains: – List of ModelMeshPart – List of Effects – Verticies for triangles that comprise this mesh VertexBuffer – Also has a bounding sphere ModelMeshPart – Represents a set of triangles that share the same materials (e.g., shader, or Effect) – Has indexes into the ModelMesh – Starting index, number of triangles, number of primitives to use from parent ModelMesh’s VertexBuffer Model ModelMesh 1 N Bone 1 N ModelMeshPart Effect 1 N 1 N VertexBuffer 1 1 Verticies 1 N Effect 1 1 StartIndex NumVerticies PrimitiveCount 1 1 1 1

35. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Working with Models in XNA Bring model into XNA project inside Visual Studio Load model into XNA via Content manager Draw model by iterating through all contained meshes

36. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Importing Models into Visual Studio Import model into Content directory – Copy.x or.fbx file, along with associated texture images, into Content directory in XNA project – Inside Visual C#, right-click on Content directory Add… Add Existing Item Select.x or.fbx file – Similar process to adding bitmap textures, etc.

37. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Loading Model via Content manager Models are loaded via the content manager – Model model = Content.Load (@”{name of my model without.x or.fbx extension}”) XNA parses the model file, and fills in verticies, textures, and effects in Model, and ModelMeshes – In XNA, this is a robust operation, big time savings – In many open source 3D game engines, model import can be a big problem – At present, typically is not safe to assume model import will work smoothly in a given 3D game engine – Need to test your tool chain

38. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides Drawing a Model Iterate through all of the meshes – Iterate through each effect for each mesh Set lighting, camera, and world for each effect – Draw each mesh using Draw() method on ModelMesh class foreach (ModelMesh mesh in model.Meshes) { foreach (BasicEffect be in mesh.Effects) { be.EnableDefaultLighting(); be.Projection = camera.projection; be.View = camera.view; be.World = world * mesh.ParentBone.Transform; } mesh.Draw(); } Examine example code from Chapter 10 of Learning XNA 3.0

39. Computer Science – Game DesignUC Santa Cruz Adapted from Jim Whitehead’s slides More on Models Explanation of parts of XNA models – http://blogs.msdn.com/shawnhar/archive/2006/1 1/20/models-meshes-parts-and-bones.aspx http://blogs.msdn.com/shawnhar/archive/2006/1 1/20/models-meshes-parts-and-bones.aspx