1 Lecture 12 Monte Carlo methods in parallel computing Parallel Computing Fall 2008

2 What is Monte Carlo? Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in simulating physical and mathematical systems. studying systems with a large number of coupled degrees of freedom, such as fluids, disordered materials, strongly coupled solids, and cellular structures (see cellular Potts model). modeling phenomena with significant uncertainty in inputs, such as the calculation of risk in business. Mathematics: e.g. evaluation of definite integrals Monte Carlo methods tend to be used when it is infeasible or impossible to compute an exact result with a deterministic algorithm (that is, intractable problems ). Monte Carlo Algorithm is a randomized algorithm that may produce incorrect results, but with bounded error probability.

3 Applications of Monte Carlo Radiation transport Operations research Nuclear criticality Design of nuclear reactors Design of nuclear weapons Statistical physics Phase transitions Wetting and growth of thin films Atomic wave functions and eigenvalues Intranuclear cascade reactions Thermodynamic properties Long chain coiling polymers Reaction kinetics Partial differential equations Large sets of linear equations Numerical integration Uncertainty analysis Development of statistical tests Cell population studies Combinatorial problems Search and optimization Signal detection WarGames

4 Example 1 - Monte Carlo Integration to Estimate Pi A simple example to illustrate Monte Carlo principals Accelerate convergence using variance reduction techniques Use if expectation values Importance sampling Adapt to a parallel environment

5 Monte Carlo Estimate of Pi

6 Serial Monte Carlo Algoritm for Pi Read N Set SumG = 0.0 Do While i < N Pick two random numbers xi and yi If (xi*xi + yi*yi £ 1) then SumG = SumG + 1 Endif Enddo Gbar = SumG / N SigGbar = Sqrt(Gbar - Gbar2) Print N, Gbar, SigGbar

7 Parallelization of Monte Carlo Integration If message passing time is negligible For same run time, error decreases by factor of Sqrt(p) For same accuracy, run time improves by a factor of p Should use same random number generator on each node (easy for homogeneous architecture)

8 Monte Carlo Estimates of Pi Number of Batch CPU Standard True Processors Size Time(sec) Deviation Error , , ,

9 Monte Carlo Estimates of Pi 1. Creating the random number generators for ( int i=0; i < num_dimension; i++) { // create r.n. generator // the seeds are different for the processors seed = (myRank + 1) * i * ; if ( seed == 0 ) rand[i] = new RandomStream(RANDOMIZE); else rand[i] = new RandomStream( seed ); }

10 Monte Carlo Estimates of Pi 2. Each processors will run num_random / numProcs random walks for ( i=myRank; i < num_random; i+=numProcs ) { // N random walks 2a. Generating random numbers for ( int i1=0; i1 < num_dimension; i1++) { ran[i1] = rand[i1]- > next(); } 2b. Calculating for integration if ( inBoundary( ran, num_dimension ) ) { myPi += f( ran, num_dimension ) * pdf( ran, num_dimension ); count++; }

11 Monte Carlo Estimates of Pi 3. Calculating Pi from each processor myPi = 4 * myPi / num_random; MPI_Reduce(&myPi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); 4. Output the results and standard deviation, sigma if (myRank == 0) { cout << "pi is = " << pi << " sigma = " << 4 * sqrt( (pi/4 - (pi/4)*(pi/4)) / num_random) << " Error = " << abs(pi - PI25DT) << endl; }

12 End Thank you!