1 Introduction to Data Parallel Architectures Sima, Fountain and Kacsuk Chapter 10 CSE462

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Basic Concept of Data Parallelism Memory Register 1Register 2 8 Bit ALU

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Basic Concept of Data Parallelism 8 bit wide Memory cells 8 bit wide Registers 8 bit ALU made of single bit ALUs

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Why is this useful? Every cell of a matrix Every pixel of an image Every record of an database Process:

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Thinking Machines l Connection Machine l Up to 65,535 processors in CM-2

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Connectivity l Want to support basic computations required at cell level –E.g. A[i,j] = (A[i-1,j] + A[i+1,j] + A[i,j-1] + A[i,j+1])/4 l To achieve this, cells can be connected in a variety of ways –Near neighbours –Tree –Graph –Pyramid –Hypercube –Multistage –Reconfigurable –Crossbar –Bus.

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Nearest Neighbours l Mapping spatially coherence data onto SIMD systems –Spatially correlated like images l Common to connect to NSEW, but diagonal has also been implemented –Applied to massively parallel systems –Scalable –Simple to implement

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Trees and Graphs l Problems expressed as graphs –E.g. database searching, model matching, expert systems, etc l No mathematically regular structure l Reconfigurability required l Binary an Quad trees common l Data bottlenecks going through roots of sub trees

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley The Pyramid l Combination of mesh and tree –Supports nearest neighbour plus (quad) tree communication –Local communications of mesh –Global communication of tree Consider example of moving data from one corner to another l Useful for data stored at multiple resolutions –Like images

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Hypercubes l 2 N processors – each of which has N links. l Fault tolerant l Shorter pathways than mesh

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Different Data Parallel Architectures l SIMD Program PE Array

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Different Data Parallel Architectures l Systolic or Pipelined DataALU 1ALU 2 ALU 3ALU 4 PE Array

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Different Data Parallel Architectures l Vectorizing Vector of Vectors Vector ALU Automatic Sequence

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Different Data Parallel Architectures l Associative and Neural Comparator Category Object Database

 David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley Principal Characteristics of data- parallel systems PropertySIMDSystolicPipelineVectorizingNeuralAssociative ProgrammabilityGoodFixed GoodPoorGood AvailabilityGoodPoor GoodPoor ScalabilityGoodFixed Good ApplicabilityWideNarrow WideNarrowWide