Slide 1

Advanced Optimization Techniques for Complex Problems Tcnicas de Optimizacin Avanzadas para Problemas Complejos TRACER:ULL - 2003 Barcelona, October 25th, 2003 http://www.tracer.ull.es TIC2002-04498-C05-05 University of La Laguna Slide 2 Outline Objectives Researchers Problems Branch and Bound and Divide and Conquer Skeletons Knapsack Problem Matrix Product Constrained two-dimensional cutting stock problem CALL and LLAC: tools for Complexity Analysis Symbolic regression Problem An analytical model for Pipeline and Master-Slave algorithms over heterogeneous clusters Resource allocation problem Prediction of the RNA Secondary Structure problem Results Slide 3 TRACER::ULL Objectives The TRACER::ULL main objective is to achieve an efficient resolution of the following complex problems by developing new optimization procedures: Constrained two-dimensional cutting stock problem Symbolic regression problem Prediction of the RNA secondary structure problem We propose the design, implementation and evaluation of solving tools using exact techniques: Divide and Conquer Branch and Bound Dynamic Programming It is an objective to provide sequential, parallel and distributed implementations for academia problems: Resource allocation problem Knapsack problem Matrix Product A second research track is related with the building of a methodology and the associated tool for the complexity and performance analysis of both sequential and parallel algorithms. Another goal is the implementation of An Internet execution systems A Problem repository Performance Analysis Web site: http://www.tracer.ull.es Slide 4 Researchers ULL Staff Coromoto Len Hernndez Isabel Dorta Gonzlez Daniel Gonzlez Morales Casiano Rodrguez Len Jess Alberto Gonzlez Martnez Foreing Rumen Andonov Students Juan Ramn Gonzlez Gonzlez Gara Miranda Valladares Mara Dolores Medina Barroso Grants Branch and Bound Dynamic Programming Performance Analysis Tools and Symbolic regression problem Divide and Conquer two dimensional cutting stock problem Prediction of the RNA secondary structure problem Slide 5 Shared Memory Branch and Bound Skeletons // shared variables {bqueue, bstemp, soltemp, data} // private variables {auxSol, high, low} // the initial subproblem is already inserted in the global shared queue while(!bqueue.empty()) { nn = bqueue.getNumberOfNodes(); nt = (nn > maxthread)?maxthread:nn; data = new SubProblem[nt]; for (int j = 0; j < nt; j++) data[j] = bqueue.remove(); set.num.threads(nt); parallel forall (i = 0; i < nt; i++) { high = data[i].upper_bound(pbm,auxSol); if ( high > bstemp ) { low = data[i].lower_bound(pbm,auxSol); if ( low > bstemp ) { // critical region // only one thread can change the value at any time bstemp = low; soltemp = auxSol; } if ( high != low ) { // critical region // just one thread can insert subproblems in the queue at any time data[i].branch(pbm,bqueue); } } bestSol = bstemp; sol = soltemp; Slide 6 0-1 Knapsack Problem The 0/1 Knapsack Problem can be stated as follows: "We have been provided with a knapsack of capacity C and with a set of N objects; p[k] and w[k] are the profit and weight associated to object k. Without exceeding the capacity of the knapsack, the objects must be inserted into the knapsack providing the maximum profit". Martello, S., Toth, P. : Knapsack Problems Algorithms and Computer Implementatios. John Wiley & Sons Ltd. (1990) Slide 7 Comparison between MPI and OpenMP skeletons Origin 3000- CIEMAT Slide 8 Distributed Branch and Bound skeleton Initialization Phase Resolution Phase Conditional Communication Message Reception Avoiding starvation Compute Best bound Propagation Work querying Ending resolution phase Solution Building Slide 9 Distributed Branch and Bound skeleton Slide 10 Slide 11 Slide 12 Slide 13 Slide 14 Matrix Product Definition: Strassen algorithm: Lets be y Slide 15 Distributed Divide and Conquer skeleton Slide 16 Two dimensional cutting stock Problem: User Interface In this problem we are given a large stock rectangle S of dimension LxW and n types of smaller rectangles (pieces) where the i-th type has dimension l i xw i. Furthermore, each problem is now to cut off from the large rectangle a set of small rectangles such that: All pieces have a fixed orientation, i.e., a piece of length l and width w is different from a piece of length w and width l (lw) All applied cuts are of guillotine type, i.e., cut that start form one edge and run parallel to the other two edges. There are at most b i rectangles of type i in the cutting plane, the demand constrain of the i-th piece. The overall profit obtained by i=1 n c i x i where x i denotes the number of rectangles of type i in the cutting patter, is maximized. Aplicacin del Proyecto Magos Slide 17 Performance: CALL & LLAC MPI PVM Standard LibrariesParallel Architectures We need a well accepted Parallel Computing Model BSP LogP...... Slide 18 CALL & LLAC Architecture Slide 19 Performance: CALL & LLAC #pragma cll mp mp[0] + mp[1]*N + mp[2]*N*N + mp[3]*N*N*N for (i = 0; i < N; i++) { for (j = 0; j < N; j++) { sum = 0; for (k = 0; k < N; k++) sum += A(i,k) * B(k,j); C(i,j) = sum; } #pragma cll end mp Square Matrix Product. A, B y C of dimension NN, Slide 20 Measuring and Predicting Performance while(!bqueue.empty()) { auxSp = bqueue.remove(); // pop a problem from the local queue high = auxSp.upper_bound(pbm,auxSol); // upper bound if ( high > bestSol ) { low = auxSp.lower_bound(pbm,auxSol); // lower bound if ( low > bestSol ) { bestSol = low; sol = auxSol; outputPacket.send(MASTER, SOLVE_TAG, bestSol, sol); } if ( high != low ) { // calculate the number of required slaves rSlaves = bqueue.getNumberOfNodes(); op.send(MASTER, BnB_TAG, high, rSlaves); inputPacket.recv(MASTER, nfSlaves, bestSol, rank {1,..., nfSlaves}); if ( nfSlaves >= 0) { auxSp.branch(pbm,bqueue); // branch and save in the local queue for i=0, nfSlaves{ // send subproblems to the assigned slaves auxSp = bqueue.remove(); outputPacket.send(rank, PBM_TAG, auxSp, bestSol, sol); } } // if nfSlaves == DONE the problem is bounded (cut) } #pragma cll code numvis++; Slide 21 How to compile? call kpr.c kpr.cll.h kpr.cll.c cc kpr kpr.c.datkpr.c.dat.1kpr.c.dat.n... EXPERIMENT: "kps" BEGIN_LINE: 115 END_LINE: 119 FORMULA: p 0 p 1 v 0 * + INFORMULA: kps[0]+kps[1]*numvis MAXTESTS: 131072 DIMENSION: 2 PARAMETERS: NUMIDENTS: 1 IDENTS: numvis OBSERVABLES: CLOCK COMPONENTS: 1 numvis POSTFIX_COMPONENT_0: 1 POSTFIX_COMPONENT_1: v 0 NUMTESTS: 1 SAMPLE: CPU NCPUS numvis CLOCK 0 1 261134.0 0.16491100 Slide 22 Number of visited Nodes Study Slide 23 Measuring and Predicting Performance int main(int argc, char ** argv) { number sol; readKnap(data); /* obj. sig., capacidad rest., beneficio */ sol = knap( 0, M, 0); printf("\nsol = ", sol); return 0; } #pragma cll code double numvis = 0.0; #pragma cll report all #pragma cll kps kps[0]*unknown(numvis) posteriori numvis #pragma cll end kps Slide 24 Symbolic Regression Problem Find the unknown complexity formula starting from the experimental data gathered by CALL. We can use Symbolic Regression: the induction of mathematical expressions on data. Rather than searching for the values of the regression constants, The object of search is a symbolic description of the system. See Scientific Discovery using Genetic Programming by Maarten Keijzer. 2001 http://www.cs.vu.nl/mkeijzer/publications/thesis/. Currently we use a fitness function that measures the error of the predictions on the asymptotic side using linear regression on a small sub-sample Aplicacin del Proyecto Magos Slide 25 Prediction of the RNA Secondary Structure Problem RNA molecule: string of n characters: R=r 1 r 2... r n such that r i {A, C, G, U} Nucleotides join to free energy: A U G U C G The iteration space is n x n triangular Dependences nonuniform: dependences among non-consecutive stages Aplicacin del Proyecto Magos E(S i+1,j-1 ) + ( r i, r j ), E( S i,j ) = min min { E(S i,k-1 ) + E(S k,j ) } i < k j Slide 26 TRACER::ULL 2003 Results Journals: Authors: Dorta, Len, Rodrguez Title: Comparing MPI and openMP Implementations of the 0-1 Knapsack Problem Journal: Parallel and Distributed Computing Practices. ISSN 1097-2803 (Accepted) Date: 2003 Authors: Blanco V., Garca L., Gonzlez J.A., Rodrguez C., Rodrguez G. Title: A Performance Model for the Analysis of OpenMP Programs Journal: Parallel and Distributed Computing Practices. ISSN 1097-2803 (Accepted) Date: 2003 Slide 27 TRACER::ULL 2003 Results International Conferences: Blanco V., Gonzlez J. A., Len C., Rodrguez C., Rodrguez G. From Complexity Analysis to Performance Analysis. Euro-Par 2003. International Conference on Parallel and Distributed Computing. Klagenfurt, Austria. 26 - 29 August 2003. Dorta I., Len C., Rodrguez C., Rojas A.Parallel Skeletons for Divide and Conquer and Branch and Bound Techniques. 11th euromicro Conference on Parallel and Network-Based Processing. ISSN 1066-6192. Genova, Italy. 5-7 February, 2003. Dorta I., Len C., Rodrguez C. A comparison between MPI and OpenMP Branch-and- Bound Skeletons. 8th International Workshop on High-Level Parallel Programming Models and Supportive Enviroments. ISBN 0-7695-1880-X. Nice, France.22 April, 2003. Dorta I., Len C., Rodrguez C., Rojas A. Parallel Skeletons. Branch-and-Bound and Divide-and-Conquer Techniques. TAM User Group Meeting 2003. Barcelona, Spain. 16 May, 2003 Dorta I., Len C., Rodrguez C., Rojas A. MPI and OpenMP implementations of Branch and Bound Skeletons. ParCo2003. Dresden, Germany. 2-5 Septiembre, 2003. Dorta I., Len C., Rodrguez C. Parallel Branch and Bound Skeletons: Message Passing and Shared Memory Implementtions. 5th International Conference on Parallel Processing and Applied Mathematics. Czestochowa, Poland. 7-10 September, 2003. Garca L., Gonzlez J.A., Gonzlez J.C., Len C., Rodrguez C., Rodrguez G. Complexity Driven Performance Analysis. 10th EuroPVM/MPI 2003. Venice, Italy. Sep 29 - Oct 2, 2003. Slide 28 TRACER::ULL 2003 Results National Conferences: Dorta I., Len C., Rodrguez C. Rodrguez, G., Rojas A. Complejidad Algortmica: de la Teora a la Prctica. JENUI03 (Jornadas de Enseanza Universitaria de la Informtica). ISBN 84-283-2845-5. Cdiz. 9-11 Julio, 2003 Gonzlez J.R., Len, C., Rodrguez C., Un esqueleto para Ramificacin y Acotacin Distribuido. XIV Jornadas De Paralelismo. Legans (Madrid). 15-17 septiembre 2003 PFC Gonzlez J. R., Esqueletos Paralelos Distribuidos. Paradigmas de Ramificacin y Acotacin y Divide y Vencers. Documento de Trabajo Interno del DEIOC: DT-03-07. Julio 2003.