1. Advanced Optimization Techniques for Complex Problems Técnicas de Optimización Avanzadas para Problemas Complejos TRACER:ULL - 2003 Barcelona, October 25th, 2003 http://www.tracer.ull.es TIC2002-04498-C05-05 University of La Laguna

2. Outline Objectives Researchers Problems Branch and Bound and Divide and Conquer Skeletons  Knapsack Problem  Matrix Product  Constrained two-dimensional cutting stock problem CALL and LLAC: tools for Complexity Analysis  Symbolic regression Problem An analytical model for Pipeline and Master-Slave algorithms over heterogeneous clusters  Resource allocation problem  Prediction of the RNA Secondary Structure problem Results

3. TRACER::ULL Objectives The TRACER::ULL main objective is to achieve an efficient resolution of the following complex problems by developing new optimization procedures:  Constrained two-dimensional cutting stock problem  Symbolic regression problem  Prediction of the RNA secondary structure problem We propose the design, implementation and evaluation of solving tools using exact techniques:  Divide and Conquer  Branch and Bound  Dynamic Programming It is an objective to provide sequential, parallel and distributed implementations for academia problems:  Resource allocation problem  Knapsack problem  Matrix Product A second research track is related with the building of a methodology and the associated tool for the complexity and performance analysis of both sequential and parallel algorithms. Another goal is the implementation of  An Internet execution systems  A Problem repository  Performance Analysis Web site: http://www.tracer.ull.es

4. Researchers ULL Staff  Coromoto León Hernández  Isabel Dorta González  Daniel González Morales  Casiano Rodríguez León  Jesús Alberto González Martínez Foreing  Rumen Andonov Students  Juan Ramón González González  Gara Miranda Valladares  María Dolores Medina Barroso Grants Branch and Bound Dynamic Programming Performance Analysis Tools and Symbolic regression problem Divide and Conquer two dimensional cutting stock problem Prediction of the RNA secondary structure problem

5. Shared Memory Branch and Bound Skeletons // shared variables {bqueue, bstemp, soltemp, data} // private variables {auxSol, high, low} // the initial subproblem is already inserted in the global shared queue while(!bqueue.empty()) { nn = bqueue.getNumberOfNodes(); nt = (nn > maxthread)?maxthread:nn; data = new SubProblem[nt]; for (int j = 0; j < nt; j++) data[j] = bqueue.remove(); set.num.threads(nt); parallel forall (i = 0; i < nt; i++) { high = data[i].upper_bound(pbm,auxSol); if ( high > bstemp ) { low = data[i].lower_bound(pbm,auxSol); if ( low > bstemp ) { // critical region // only one thread can change the value at any time bstemp = low; soltemp = auxSol; } if ( high != low ) { // critical region // just one thread can insert subproblems in the queue at any time data[i].branch(pbm,bqueue); } } bestSol = bstemp; sol = soltemp;

6. 0-1 Knapsack Problem The 0/1 Knapsack Problem can be stated as follows: "We have been provided with a knapsack of capacity C and with a set of N objects; p[k] and w[k] are the profit and weight associated to object k. Without exceeding the capacity of the knapsack, the objects must be inserted into the knapsack providing the maximum profit". Martello, S., Toth, P. : Knapsack Problems Algorithms and Computer Implementatios. John Wiley & Sons Ltd. (1990)

7. Comparison between MPI and OpenMP skeletons Origin 3000- CIEMAT

8. Distributed Branch and Bound skeleton Initialization Phase Resolution Phase  Conditional Communication  Message Reception  Avoiding starvation  Compute  Best bound Propagation  Work querying  Ending resolution phase Solution Building

9. Distributed Branch and Bound skeleton

14. Matrix Product Definition: Strassen algorithm: Lets be y

15. Distributed Divide and Conquer skeleton

16. Two dimensional cutting stock Problem: User Interface In this problem we are given a large stock rectangle S of dimension LxW and n types of smaller rectangles (pieces) where the i-th type has dimension l i xw i. Furthermore, each problem is now to cut off from the large rectangle a set of small rectangles such that:  All pieces have a fixed orientation, i.e., a piece of length l and width w is different from a piece of length w and width l (l≠w)  All applied cuts are of guillotine type, i.e., cut that start form one edge and run parallel to the other two edges.  There are at most b i rectangles of type i in the cutting plane, the demand constrain of the i-th piece.  The overall profit obtained by Σ i=1 n c i x i where x i denotes the number of rectangles of type i in the cutting patter, is maximized. Aplicación del Proyecto Magos

17. Performance: CALL & LLAC MPI PVM Standard LibrariesParallel Architectures We need a well accepted Parallel Computing Model BSP LogP......

18. CALL & LLAC Architecture

19. Performance: CALL & LLAC #pragma cll mp mp[0] + mp[1]*N + mp[2]*N*N + mp[3]*N*N*N for (i = 0; i < N; i++) { for (j = 0; j < N; j++) { sum = 0; for (k = 0; k < N; k++) sum += A(i,k) * B(k,j); C(i,j) = sum; } #pragma cll end mp Square Matrix Product. A, B y C of dimension N×N,

20. Measuring and Predicting Performance while(!bqueue.empty()) { auxSp = bqueue.remove(); // pop a problem from the local queue high = auxSp.upper_bound(pbm,auxSol); // upper bound if ( high > bestSol ) { low = auxSp.lower_bound(pbm,auxSol); // lower bound if ( low > bestSol ) { bestSol = low; sol = auxSol; outputPacket.send(MASTER, SOLVE_TAG, bestSol, sol); } if ( high != low ) { // calculate the number of required slaves rSlaves = bqueue.getNumberOfNodes(); op.send(MASTER, BnB_TAG, high, rSlaves); inputPacket.recv(MASTER, nfSlaves, bestSol, rank {1,..., nfSlaves}); if ( nfSlaves >= 0) { auxSp.branch(pbm,bqueue); // branch and save in the local queue for i=0, nfSlaves{ // send subproblems to the assigned slaves auxSp = bqueue.remove(); outputPacket.send(rank, PBM_TAG, auxSp, bestSol, sol); } } // if nfSlaves == DONE the problem is bounded (cut) } #pragma cll code numvis++;

21. How to compile? call kpr.c kpr.cll.h kpr.cll.c cc kpr kpr.c.datkpr.c.dat.1kpr.c.dat.n... EXPERIMENT: "kps" BEGIN_LINE: 115 END_LINE: 119 FORMULA: p 0 p 1 v 0 * + INFORMULA: kps[0]+kps[1]*numvis MAXTESTS: 131072 DIMENSION: 2 PARAMETERS: NUMIDENTS: 1 IDENTS: numvis OBSERVABLES: CLOCK COMPONENTS: 1 numvis POSTFIX_COMPONENT_0: 1 POSTFIX_COMPONENT_1: v 0 NUMTESTS: 1 SAMPLE: CPU NCPUS numvis CLOCK 0 1 261134.0 0.16491100

22. Number of visited Nodes Study

23. Measuring and Predicting Performance int main(int argc, char ** argv) { number sol; readKnap(data); /* obj. sig., capacidad rest., beneficio */ sol = knap( 0, M, 0); printf("\nsol = ", sol); return 0; } #pragma cll code double numvis = 0.0; #pragma cll report all #pragma cll kps kps[0]*unknown(numvis) posteriori numvis #pragma cll end kps

24. Symbolic Regression Problem Find the unknown complexity formula starting from the experimental data gathered by CALL. We can use Symbolic Regression: the induction of mathematical expressions on data. Rather than searching for the values of the regression constants, The object of search is a symbolic description of the system. See Scientific Discovery using Genetic Programming by Maarten Keijzer. 2001 http://www.cs.vu.nl/˜mkeijzer/publications/thesis/. Currently we use a fitness function that measures the error of the predictions “on the asymptotic side” using linear regression on a small sub-sample Aplicación del Proyecto Magos

25. Prediction of the RNA Secondary Structure Problem RNA molecule: string of n characters: R=r 1 r 2... r n such that r i  {A, C, G, U} Nucleotides join to free energy: A  U G  U C  G The iteration space is n x n triangular Dependences nonuniform: dependences among non-consecutive stages Aplicación del Proyecto Magos E(S i+1,j-1 ) +  ( r i, r j ), E( S i,j ) = min min { E(S i,k-1 ) + E(S k,j ) } i < k  j

26. TRACER::ULL 2003 Results Journals:  Authors: Dorta, León, Rodríguez Title: Comparing MPI and openMP Implementations of the 0-1 Knapsack Problem Journal: Parallel and Distributed Computing Practices. ISSN 1097-2803 (Accepted) Date: 2003  Authors: Blanco V., García L., González J.A., Rodríguez C., Rodríguez G. Title: A Performance Model for the Analysis of OpenMP Programs Journal: Parallel and Distributed Computing Practices. ISSN 1097-2803 (Accepted) Date: 2003

27. TRACER::ULL 2003 Results International Conferences:  Blanco V., González J. A., León C., Rodríguez C., Rodríguez G. “From Complexity Analysis to Performance Analysis”. Euro-Par 2003. International Conference on Parallel and Distributed Computing. Klagenfurt, Austria. 26 - 29 August 2003.  Dorta I., León C., Rodríguez C., Rojas A.”Parallel Skeletons for Divide and Conquer and Branch and Bound Techniques”. 11th euromicro Conference on Parallel and Network-Based Processing. ISSN 1066-6192. Genova, Italy. 5-7 February, 2003.  Dorta I., León C., Rodríguez C. “A comparison between MPI and OpenMP Branch-and- Bound Skeletons”. 8th International Workshop on High-Level Parallel Programming Models and Supportive Enviroments. ISBN 0-7695-1880-X. Nice, France.22 April, 2003.  Dorta I., León C., Rodríguez C., Rojas A. “Parallel Skeletons. Branch-and-Bound and Divide-and-Conquer Techniques”. TAM User Group Meeting 2003. Barcelona, Spain. 16 May, 2003  Dorta I., León C., Rodríguez C., Rojas A. “MPI and OpenMP implementations of Branch and Bound Skeletons”. ParCo2003. Dresden, Germany. 2-5 Septiembre, 2003.  Dorta I., León C., Rodríguez C. “Parallel Branch and Bound Skeletons: Message Passing and Shared Memory Implementtions”. 5th International Conference on Parallel Processing and Applied Mathematics. Czestochowa, Poland. 7-10 September, 2003.  García L., González J.A., González J.C., León C., Rodríguez C., Rodríguez G. “Complexity Driven Performance Analysis”. 10th EuroPVM/MPI 2003. Venice, Italy. Sep 29 - Oct 2, 2003.

28. TRACER::ULL 2003 Results National Conferences:  Dorta I., León C., Rodríguez C. Rodríguez, G., Rojas A. “Complejidad Algorítmica: de la Teoría a la Práctica”. JENUI’03 (Jornadas de Enseñanza Universitaria de la Informática). ISBN 84-283-2845-5. Cádiz. 9-11 Julio, 2003  González J.R., León, C., Rodríguez C., ”Un esqueleto para Ramificación y Acotación Distribuido”. XIV Jornadas De Paralelismo. Leganés (Madrid). 15-17 septiembre 2003 PFC  González J. R., “Esqueletos Paralelos Distribuidos. Paradigmas de Ramificación y Acotación y Divide y Vencerás”. Documento de Trabajo Interno del DEIOC: DT-03-07. Julio 2003.