Presentation is loading. Please wait.

Presentation is loading. Please wait.

Offline Adaptation Using Automatically Generated Heuristics Frédéric de Mesmay, Yevgen Voronenko, and Markus Püschel Department of Electrical and Computer.

Similar presentations


Presentation on theme: "Offline Adaptation Using Automatically Generated Heuristics Frédéric de Mesmay, Yevgen Voronenko, and Markus Püschel Department of Electrical and Computer."— Presentation transcript:

1 Offline Adaptation Using Automatically Generated Heuristics Frédéric de Mesmay, Yevgen Voronenko, and Markus Püschel Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA Marius Fehr, December 7 th

2 Online vs. Offline Adaptive Libraries 2 Online Adaptive Offline Adaptive Machine Learning d = dft(n) Search generated at Installation d(X,Y) – possibly unbounded initialization time – impractical for constantly changing problem specifications + bounded initialization time + no search if input specifications change

3 3 Motivation Background Adaptation Process Results

4 Statistical Classifier – C4.5 4 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU features ˃Based on entropy, the measure of uncertainty ˃Feature with smallest entropy becomes root

5 DFT – Discrete Fourier Transforms ˃The Fourier Transform is a Linear Transform: 5 X X Y Y DFT n = *

6 FFT – Fast Fourier Transforms ˃Divide and Conquer Algorithms 6 Source: Lecture Slides :“How to Write Fast Numerical Code “ (ETH, CS), Markus Püschel

7 FFTW – Search space 7 dft ( 128 ) These decisions can be optimized! Radix 8 dft_strided (16, 8) dft_scaled (8, 16) Radix 4 No ˃Features :Problem size and stride ˃Decisions:Use of base cases and choice of radix Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

8 FFTW – Advanced Implementations 8 Lots of decisions! Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

9 9 Motivation Background Adaptation Process Results

10 Offline Adaptation Process 10 C4.5 Search x x x x x x x x x x x x x x Heuristic featuresdecisions sizestrideuse base caseradix 128-no8 16-no4 4-yes- 168no4 ………… Trainings Set

11 Offline Adaptation Process 11 Exploration Statistical Classification Verification Combination Online Adaptive Library Offline Adaptive Library Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

12 Exploration ˃“Creates a table for the statistical classifier to work with” 12 Exploration Statistical Classification Verification Combination Search x x x x x x x x x x x x x x featuresdecisions SizeStrideUse Base caseRadix 128-No8 16-No4 4-Yes- 168No4 ………… Trainings Set

13 Statistical Classification ˃Computes decision trees ˃Uses a modified version of C4.5 ˃Provides “hints” based on library functionality 13 Exploration Statistical Classification Verification Combination Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

14 Hinting ˃Problem:Sometimes C4.5 has not enough information to make wise decisions ˃Choice of radix: ˃Performance depends strongly on the prime factorization ˃Multiple of 2 and 3 show very different behavior but are heavily interleaved ˃Solution:Provide additional features to C Exploration Statistical Classification Verification Combination n n < … … … 58 > 59 Radix ?

15 Hinting 15 Exploration Statistical Classification Verification Combination Radix 12 Radix 3 Radix 18 Radix 6 Radix 4 nfactor( 2, n ) nfactor( 3, n ) n

16 16 Exploration Statistical Classification Verification Combination Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

17 Verification 17 Exploration Statistical Classification Verification Combination 24 = 2 * 2 * 2* 3 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

18 Combination ˃Inserts decision trees into the library as heuristics 18 Exploration Statistical Classification Verification Combination Online Adaptive d = dft(n) Search d(X,Y) Offline Adaptive

19 19 Motivation Background Adaptation Process Results

20 2-Powers – Competitiveness 20 Platform: 2 x dual core 3GHz Intel Xeon 5160 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

21 Learning and Generating Heuristics 21 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

22 Mixed Sizes - Competitiveness 22 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

23 ˃Pro ˃Improvement of usability ˃Only small performance penalty (for DFT) ˃Entirely automatic method ˃Applicable to other problem domains / libraries ˃Computer generation of offline adaptive library, directly from algorithm specification (together with Spiral) ˃Contra ˃Performance depends strongly on choice of training set Pro vs. Contra Algorithm Specification SPIRAL this paper Adaptive Offline Library 23

24 Questions ? 24

25 Questions ? 25

26 Online vs. Offline Adaptive Libraries 3 Library TypeNon – adaptiveOnline adaptiveOffline adaptive PrototypeIPPFFTWthis paper Interface Initialization cost Computation cost Adaptation mechanism -online (planer at runtime) offline (at installation time) problem changes-rerun planer- platform changesrebuyrerun planerreinstall 26 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

27 Math behind C4.5 27


Download ppt "Offline Adaptation Using Automatically Generated Heuristics Frédéric de Mesmay, Yevgen Voronenko, and Markus Püschel Department of Electrical and Computer."

Similar presentations


Ads by Google