Easy Optimization Problems, Relaxation, Local Processing for a single variable

The advantages of multileveling  Linear running time  Provided a “good coarsening” is performed: 1.The number of variables is reduced in such a way that preserves the essence (skeleton) of the graph => an easier problem 2.Enables processing in different scales, moves which are not likely to happen systematically in a “flat” approach  A better GLOBAL solver

General 1D Arrangement Problems A graph G(v,  a ij

General 1D Arrangement Problems E(x)=  i j a i j | x i -x j | p x i = v i /2 +  k:  k)<  i) v k ij xixi xjxj From the graph To the arrangement a ij

The complexity of pointwise relaxation for P=2 Go over all variables in lexicographic order, put x i at the weighted average location of its graph neighbors Problem: Does not preserve the volume demands! Reinforce volume demands at the end of each sweep  If the reinforcement is done after every variable, the complexity will be quadratic and not linear !  Sorting of x i is O(nlogn), however, usually logn<C, where C is the constant of the linear complexity  If the ‘sort’ is too slow, use bucketing instead

Different types of relaxation  Variable by variable relaxation – strict minimization  Changing a small subset of variables simultaneously – Window strict minimization relaxation  Stochastic relaxation – may increase the energy – should be followed by strict minimization

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model

Exc#1: 2D Ising spins exercise  Minimize  Periodic boundary condition  Initialize randomly: with probability.5 1.Go over the grid in lexicographic order, for each spin choose 1 or -1 whichever minimizes the energy (choose with probability ½ when the two possibilities have the same energy) until no changes are observed. 2. Repeat 3 times for each of the 4 possibilities of (h 1,h 2 ). 3. Is the global minimum achievable? 4. What local minima do you observe?

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2  General functional : P=1, P>2

Exc#2: Pointwise relaxation for P=1  Minimize  Pick a variable, fix all at  Minimize  Find the optimal location for

Pointwise relaxation for P=6  Minimize  Pick a variable, fix all at  Minimize  Find the roots (zeros) of

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2  General functional : P=1, P>2  Newton’s Method

Newton’s Method (Newton-Raphson)  Geometry  Taylor expansion Starting close enough to the root results with a very fast convergence  What is “close enough”?  May even diverge or oscillate  Verify local reduction in E

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2  General functional : P=1, P>2  Newton’s Method  Verify local reduction in E  Numerical derivatives

Numerical derivatives  Newton’s Method :  Calculate numerically

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2  General functional : P=1, P>2  Newton’s Method  Verify local reduction in E  Numerical derivatives  Steepest descent

Steepest descent  Level sets

Steepest descent  Level sets E(x,y)=x 2 +y 2

Steepest descent  Level sets E(x,y)=x 2 +y 2 c1c1 c2c2 c 1 < c 2

Steepest descent  Level sets  The gradient at a point is perpendicular to the level set and is directed towards the maximal rate of increase in the energy  Vector field of gradients

Steepest descent The vector field of the gradient of E(x,y)=x 2 +y 2 At every point it is perpendicular to the level set c1c1 c2c2 c 1 < c 2

Steepest descent  Level sets  The gradient at a point is perpendicular to the level set and is directed towards the maximal rate of increase in the energy  Vector field of gradients => Choose the opposite direction of the gradient as the direction for maximal decrease of the energy How much should you go in this direction?

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2  General functional : P=1, P>2  Newton’s Method  Verify local reduction in E  Numerical derivatives  Steepest descent  Line search

Line search  Starting at some  Minimize along  Exact minimization: solve  Guess an and use backtracking  Quadratic approximation: Choose, then, draw a parabola through the 3 points and find its minimum  Verify local reduction in E  If not - choose the available minimum

Exc#3: Steepest descent exercise For at Find the steepest descent direction Compare its analytical and numerical calculations Choose 2 small steps in this direction Draw a parabola through the 3 points Find the minimum of the parabola Verify reduction in the energy Find a step that increases the energy

An example of a single node relaxation for the placement problem

The placement problem Given a hypergraph: 1. A list of nodes each with its length and pins’ location 2. A list of lists of subsets of nodes - hyperedges

The hypergraph for a microchip

The placement problem Given a hypergraph: 1. A list of nodes each with its length and pins’ location 2. A list of lists of subsets of nodes - hyperedges  Minimize the sum of all wires approximated by the half Bounding Box of each hyperedge

Bounding Box The bounding box Pins’ locations

The placement problem Given a hypergraph: 1. A list of nodes each with its length and pins’ location 2. A list of lists of subsets of nodes - hyperedges  Minimize the sum of all wires approximated by the half Bounding Box of each hyperedge  Approximate the hypergraph by a graph and the Bounding Box by a quadratic functional

From hypergraph to graph Add a virtual node at the center of mass of the nodes belonging to an hyperedge

From hypergraph to graph Add a virtual node at x 0, the center of mass of the nodes The resulting graph is x1x1 x3x3 x2x2 x4x4 x 0 =(x 1 + x 2 + x 3 + x 4 ) /4 E(x)=  i (x i - x 0 ) 2, i=1,…,4 By eliminating x 0 : E(x)=  ij (x i - x j ) 2 /4, i,j=1,…,4

From hypergraph to graph E(x)=  ij (x i - x j ) 2 /4, i,j=1,…,4 x1x1 x3x3 x2x2 x4x4 A hyperedge with n nodes contributes n(n-1)/2 quadratic terms with weight 1/n to E(x) Creates a clique of connections

The placement problem Given a hypergraph: 1. A list of nodes each with its length and pins’ location 2. A list of lists of subsets of nodes - hyperedges  Minimize the sum of all wires approximated by the half Bounding Box of each hyperedge  Approximate the hypergraph by a graph and the Bounding Box by a quadratic function The placement has 2 phases: global and detailed  Use the original definition towards the detailed

Approximations for the placement problem  Given a hypergraph => translate it to a graph  The nodes are now connected at their center of mass (not at the pins) with straight lines (not rectilinear connections)  The used energy function is quadratic (not the bounding box)  Towards the end of the global placement and definitely at the (discrete) detailed placement use the original definition of the problem not the approximations

Data structure For each node in the graph keep 1.A list of all the graph’s neighbors: for each neighbor keep a pair of index and weight 2.… 3.… 4.Its current placement 5.The unique square in the grid the node belongs to For each square in the grid keep 1.A list of all the nodes which are mostly within  Defines the current physical neighborhood

The augmented E(x i,y i ) to be minimized For node i, fix all other nodes at their current and minimize the augmented functional where j are the nodes in the 3x3 window of squares around the square which includes i

The augmented E(x i,y i ) to be minimized For node i, fix all other nodes at their current and minimize the augmented functional where j are the nodes in the 3x3 window of squares around the square which includes i  How can the “steepest descent direction” be found?

The “steepest descent”

The augmented E(x i,y i ) to be minimized For node i, fix all other nodes at their current and minimize the augmented functional where j are the nodes in the 3x3 window of squares around the square which includes i  How can the “steepest descent direction” be found?  Use numerical “discrete derivatives”  For simplicity calculate each direction separately!  This is a numerical discrete line search minimization

Move node to the right

Changes in the energy and overlap

X-direction “discrete derivatives”  For node, fix all other nodes at their current  Current overlap  Calculate  Calculate optimal change by quadratic approximation  Check that the energy has indeed decreased  Update the data structure at the end of the sweep

Problems  Slowly converging  Minimization of one variable may conflict and influence other variables => instead of a single variable at a time, use a small subset Window minimization