1. Introduction to CUDA Programming Textures Andreas Moshovos Winter 2009 Some material from: Matthew Bolitho’s slides

2. Memory Hierarchy overview Registers –Very fast Shared Memory –Very Fast Local Memory –400-600 cycles Global Memory –400-600 cycles Constant Memory –400-600 cycles Texture Memory –400-600 cycles –8K Cache

3. What is Texture Memory A block of read-only memory shared by all multi- processors –1D, 2D, or 3D array –Texels: Up to 4-element vectors –x, y, z, w Reads from texture memory can be “samples” of multiple texels Slow to access –several hundred clock cycle latency But it is cached: –8KB per multi-processor –Fast access if cache hit Good if you have random accesses to a large read-only data structure

4. Overview: Benefits & Limitations of CUDA textures Texture fetches are cached –Optimized for 2D locality We’ll talk about this at the end Addressing: –1D, 2D, or 3D Coordinates: –integer or normalized –Fewer addressing calculations in code Provide filtering for free Free out-of-bounds handling: wrap modes –Clamp to edge / warp Limitations of CUDA textures: –Read-only from within a kernel

5. Texture Abstract Structure A 1D, 2D, or 3D array. Example 4x4: Values Set by the Program

6. Regular Indexing Indexes are floating point numbers –Think of the texture as a surface as opposed to a grid for which you have a grid of samples Not there

7. Normalized Indexing NxM Texture: –[0,1.0) x [0.0, 1.0) indexes (0.0,0.0) (1.0,1.0) (0.5,0.5) Convenient if you want to express the computation in size-independent terms

8. How to think about program values Values

9. What Value Does a Texture Reference Return? Nearest-Point Sampling –Comes for “free” –Elements must be floats

10. Nearest-Point Sampling In this filtering mode, the value returned by the texture fetch is –tex(x) = T[i] for a one-dimensional texture, –tex(x, y) = T[i, j] for a two-dimensional texture, –tex(x, y, z) = T[i, j, k] for a three-dimensional texture, where i = floor(x), j = floor( y), and k = floor(z).

11. Nearest-Point Sampling: 4-Element 1D Texture Behaves more like a conventional array

12. Another Filtering Option Linear Filtering See Appendix D of the Programming Guide

13. Linear-Filtering Detail Good luck with this one: Effectively the value read is a weighted average of all neighboring texels

14. Linear-Filtering: 4-Element 1D Texture

15. Dealing with Out-of-Bounds References Clamping –Get’s stuck at the edge i < 0  actual i = 0 i > N -1  actual i = N -1 Warping –Warps around actual i = i MOD N Useful when texture is a periodic signal

16. Texture Addressing Explained

17. Texels Texture Elements –All elemental datatypes Integer, char, short, float (unsigned) –CUDA vectors: 1, 2, or 4 elements char1, uchar1, char2, uchar2, char4, uchar4, short1, ushort1, short2, ushort2, short4, ushort4, int1, uint1, int2, uint2, int4, uint4, long1, ulong1, long2, ulong2, long4, ulong4, float1, float2, float4,

18. Programmer’s view of Textures Texture Reference Object –Use that to access the elements –Tells CUDA what the texture looks like Space to hold the values –Linear Memory (portion of memory) Only for 1D textures –CUDA Array Special CUDA Structure used for Textures –Opaque Then you bind the two: –Space and Reference

19. Texture Reference Object –texture texRef; Type = texel datatype Dim = 1, 2, 3 ReadMode: –What values are returned cudaReadModeElementType –Just the elements  What you write is what you get cudaReadModeNormalizedFloat –Works for chars and shorts (unsigned) –Value normalized to [0.0, 1.0]

20. CUDA Containers: Linear Memory Bound to linear memory –Global memory is bound to a texture CudaMalloc() –Only 1D –Integer addressing –No filtering, no addressing modes –Return either element type or normalized float

21. CUDA Containers: CUDA Arrays Bound to CUDA arrays –CUDA array is bound to a texture –1D, 2D, or 3D –Float addressing size-based, normalized –Filtering –Addressing modes clamping, warping –Return either element type or normalized float

22. CUDA Texturing Steps Host (CPU) code: –Allocate/obtain memory global linear, or CUDA array –Create a texture reference object Currently must be at file-scope –Bind the texture reference to memory/array –When done: Unbind the texture reference, free resources Device (kernel) code: –Fetch using texture reference –Linear memory textures: tex1Dfetch() –Array textures: tex1D(), tex2D(), tex3D()

23. Texture Reference Parameters Immutable parameters @ compile-time Specified at compile time –Type: texel type Basic int, float types CUDA 1-, 2-, 4-element vectors –Dimensionality: 1, 2, or 3 –Read Mode: cudaReadModeElementType cudaReadModeNormalizedFloat –valid for 8- or 16-bit ints –returns [-1,1] for signed, [0,1] for unsigned

24. Texture Reference Mutable Parameters Mutable parameters Can be changed at run-time –only for array-textures –Normalized: non-zero = addressing range [0, 1] –Filter Mode: cudaFilterModePoint cudaFilterModeLinear –Address Mode: cudaAddressModeClamp cudaAddressModeWrap

25. Example: Linear Memory // declare texture reference (must be at file-scope) Texture texRef; // Type, Dimensions, return value normalization // set up linear memory on Device unsigned short *dA = 0; cudaMalloc ((void**)&dA, numBytes); // Copy data from host to device cudaMempcy(dA, hA, numBytes, cudaMemcpyHostToDevice); // bind texture reference to array cudaBindTexture(NULL, texRef, dA, numBytes);

26. How to Access Texels In Linear Memory Bound Textures Type tex1Dfetch(texRef, int x); Where Type is the texel datatype Previous example: –Unsigned short value = tex1Dfetch (texRef, 10) –Returns element 10 You can write to the memory holding the texture  dA allocated with cudaMalloc –Bad idea  no hardware guarantees

27. CUDA Array Type Got to specify two things: –Channel format –Dimensions CudaMallocArray –2D arrays CudaMallocArray3D –3D arrays Management functions: –cudaMallocArray, cudaFreeArray, –cudaMemcpyToArray, cudaMemcpyFromArray,...

28. Channel Descriptors What data appears on each element –Think of images for example –Every element is an RBG value cudaChannelFormatDesc structure –int x, y, z, w: parts for each component Number of bits: e.g., 8 –enum cudaChannelFormatKind – one of: cudaChannelFormatKindSigned cudaChannelFormatKindUnsigned cudaChannelFormatKindFloat –Some predefined constructors: cudaCreateChannelDesc (void); Management functions: –cudaMallocArray, cudaFreeArray, –cudaMemcpyToArray, cudaMemcpyFromArray,...

29. Example Host Code for 2D array // declare texture reference (must be at file-scope) Texture texRef; // set up the CUDA array cudaChannelFormatDesc cf = cudaCreateChannelDesc (); cudaArray *texArray = 0; cudaMallocArray(&texArray, &cf, dimX, dimY); cudaMempcyToArray(texArray, 0,0, hA, numBytes, cudaMemcpyHostToDevice); // specify mutable texture reference parameters texRef.normalized = 0; texRef.filterMode = cudaFilterModeLinear; texRef.addressMode = cudaAddressModeClamp; // bind texture reference to array cudaBindTextureToArray(texRef, texArray);

30. Accessing Texels Type tex1D(texRef, float x); Type tex2D(texRef, float x, float y); Type tex3D(texRef, float x, float y, float z);

31. At the end cudaUnbindTexture (texRef)

32. Dimension Limits In Elements not bytes –In CUDA Arrays: 1D: 8K 2D: 64K x 32K 3D: 2K x 2K x 2K –If in linear memory: 2^27 That’s 128M elements Floats: –128M x 4 = 512MB Not verified: Info from: Cyril Zeller of NVIDIA –http://forums.nvidia.com/index.php?showtopic=29545 &view=findpost&p=169592

33. Textures are Optimized for 2D Locality Regular Array Allocation –Row-Major Because of Filtering –Neighboring texels –Accessed close in time

34. Textures are Optimized for 2D Locality

35. Using Textures Textures are read-only –Within a kernel A kernel can produce an array –Cannot write CUDA Arrays Then this can be bound to a texture for the next kernel Linear Memory can be copied to CUDA Arrays –cudaMemcpyFromArray() Copies linear memory array to a CudaArray –cudaMemcpyToArray() Copies CudaArray to linear memory array

36. An Example http://www.mmm.ucar.edu/wrf/WG2/GPU/Scala r_Advect.htmhttp://www.mmm.ucar.edu/wrf/WG2/GPU/Scala r_Advect.htm GPU Acceleration of Scalar Advection

37. Cuda Arrays Read the CUDA Reference Manual Relevant functions are the ones with “Array” in it Remember: –Array format is opaque Pitch: –Padding added to achieve good locality –Some functions require this pitch to be passed as a an argument –Prefer those that use it from the Array structure directly