…AND THE END OF MULTICORE SCALING

…AND THE END OF MULTICORE SCALING
DARK Talk by PHILIPP OTT DARK SILICON… …AND THE END OF MULTICORE SCALING L&G + Prof Mutlu My Name… Helloween [Click] Lady’s and Gentlemen – dear Prof Mutlu. My Name is Philipp Ott, and I thought, since it’s Helloween i gonna tell you a horror-story. [CLICK] A story about Dark Silicon and the end of Multicore Scaling Hadi Esmaeilzadeh Emily Blem Renée St. Amant Karthikeyan Sankaralingam Doug Burger University of Washington University of Wisconsin-Madison The University of Texas at Austin Microsoft Research

Dark Silicon and the End of Multicore Scaling
Original Header Isca 2011 Hadi Esmael zade of U Washington Wisconsin-Madison Texas at Austin Microsoft Research Right here you see the Header of the Original Paper, as published at Isca 2011. Written by Hadi Esmaeilzadeh from the University of Washington and other Contributers from The Universities of Wisconsin-Madison and Texas, Austin – not to forgett Microsoft Research. Dark Silicon and the End of Multicore Scaling

EXECUTIVE SUMMARY Problem: Dennard- and multicore-scaling don’t work anymore Transistor underutilisation is underrated Goal: Predict how technology-scaling will affect transistor-underutilisation Method: Model future chip-development Calculate future transistor underutilisation Result: Power- & multicore-scaling won’t sustain Moore’s Law due to transistor underutilisation I now gonna quickly summarize the contents of the paper. [CLICK] The problem it tries to cover is that Denard and Multicore Scaling don’t work anymore. Furthermore there is widespread Transistor-Underutilisation which (ass we’ll see) is wildly underrated [CLICK] The goal of the paper is to find what causes this problems and also predict how this problems might develop and change in the future. To reach their goal, the authors created an model for future chip-development, from which the calculated future transistor-underutilisation and other factors In the end the paper showerd that, Power- & multicore-scaling won’t sustain Moore’s Law due to transistor under-utilisation Dark Silicon and the End of Multicore Scaling

I. II. III. OUTLINE THE PROBLEM/ NOVELTY THE PROBLEM/ NOVELTY
THE MODEL THE RESULTS THE GOOD / THE IMPROVABLE Insides of Paper Around Paper Your Call to Action May I present you the Outline for my Talk today: In the first part we’ll be talking in-depth about the paper, the problems, the model it creates and it’s results [CLICK] In a second segment I’ll tell you what I thought about the paper and what you should take-away from it And at the end, as usual you’ll be given the opportunity to ask questions and to participate in an Discussion Let’s start by looking at the problem, and the novelty the paper brings to the table TAKE-AWAYS / THE FUTURE QUESTIONS / DISCUSSION Dark Silicon and the End of Multicore Scaling

The Problem / The Novelty
Dennard scaling: Broke down ~2006 Solution: Multicore-scaling Exploit parallelism by adding more cores Keeping Moore’s Law alive T Moores Law forever? [Click] Enabler Double T same A Graph until 2005 [Click] Graph 2006 Heat->Power->Frequency->Performance [Click] as in Graph -> Multicore 5 years later -> wondering…. [Click] How many of you still believe that Moores law will be sustained forever? Yeah – neither did the author’s of this paper. One of the big enablers of Moores Law was Denard-Scaling, which more or less meant, That even if you pack twice as many, smaller transistors on a chip, you’d still need only the same power for the same Area. As you see in this Graph until 2005 the number of Transistors , the Performance and the clock-frequency kept happily increasing Now, bad new’s incoming: At around 2006 Micro-Architect’s had to accept that this was no longer possible, mostly due to current-leakage and therefore increased heat in the chip. So the Transistor-Count, the Performance and the Clock-Frequency started to stagnate. As you see from the black line at the bottom, this was the Moment when the Industry started adding more Cores to their Chips. And it seemed to be the solution – to exploit parallellism to keep Moores law alive. And then, aprrox 5 years later, in 2011, People started wondering… [CLICK] Source: Benchmarking Adiabatic Quantum Optimization for Complex Network Analysis Dark Silicon and the End of Multicore Scaling

We aren’t getting the performance-gains, we’d expect – but why?
THE INVISIBLE ENEMY: DARK SILICON We arent getting… [Click] Paper Steps in, uncovering… But what is DS? [Click] We aren’t getting the Performance-Gains we’d expect – but why? This is where this Paper from 2011 steps in – by uncovering an invisible Enemy lurking in the shadows: [CLICK] Dark Silicon But what does Dark Silicon mean? Dark Silicon and the End of Multicore Scaling

Dark Silicon – […]the amount of circuitry of an integrated circuit that cannot be powered-on at the nominal operating voltage for a given thermal design power (TDP) constraint Wikipedia Dark Silicon – Transistors which suffer from under-utilization Wikipedia read Wall [Click] Paper broader Def. If you’re first impulse would now be to check on wikipedia – this is what you’d find: Dark Silicon is the amount of circuitry of an integrated circuit that cannot be powered-on at the nominal operating voltage for a given thermal design power (TDP) constraint. This Paper uses a broader definition as Dark Silicon simply specifies Transistors that suffer from under-utilisation. Dark Silicon and the End of Multicore Scaling

An analogy Weather-forecast for performance-scaling Based on models Explain why the weather is getting rought MON TUE WED THU FRI SAT SUN ? Achieve Weather Forecast [Click] Wind, Temperature, Pressure -> Power, Frequency, Workloads [Click] Weather cloudy or silicon dark [Click] Technology help Sun shine [Click] To explain how this paper works, i’ll give you a little analogy: You could say it’s like a weather-forecast – but for Performance-Scaling of Processors As in the Art of Weather-Prediction – it works on Models - but instead of Models for Wind Temperature and pressure – you have Models for Transistor-Size, Power-Consumption and Frequency. We are not only interessted what the Weather will be at Saturday – we also wan’t to know why it is getting so bad. And we might not already know what happens on Sunday – but we may aswell discuss this afterwards  Dark Silicon and the End of Multicore Scaling

I. II. III. OUTLINE THE PROBLEM/ NOVELTY THE MODEL THE MODEL
THE RESULTS THE GOOD / THE IMPROVABLE - Talk about this model So now i gonna explain you how this model works TAKE-AWAYS / THE FUTURE QUESTIONS / DISCUSSION Dark Silicon and the End of Multicore Scaling

THE MODEL This model represents a best-case scenario! Scaling-Models
Multicore Device Core } Scaling-Models sub-models Aspects scaling future Cartesian of Model-Configurations For each future Tech Gen [Click] #C, MC S-Up, %DS (u.u.) [Click] BC-Scenario, Performance, DS [Click] Through these 3 Submodels To make accurate forecast, we combine multiple Prediction-Models into one. You might aswell think of it as an huge cartesian product of the three Model-Set’s What we wan’t to get out of these Models are Predictions for the Optimal Nr of Cores and it’s resulting Multicore SpeedUp – furthermore we wan’t to detect the percentage of Transistors which suffer from underutilization therefore being Dark Silicon. At this point it’s probably a good Idea to emphasize that all Predictions these models make – are inherently optimistic. This means that future Performance is likely to be worse then predicted and Dark Silicon wider spread. I now gonna explain each sub-model one-by-one, starting with the Model for Device-Scaling { Predictions Optimal # of Cores Multicore Speed-Up % of Dark Silicon } Dark Silicon and the End of Multicore Scaling Device Scaling Core Scaling Multicore Scaling

Device Model Multicore Device Core Normalized Scaling Factor Normalized Scaling Factor Optimal # of Cores Multicore Speed-Up % of Dark Silicone T development Christal Ball -> Renomated Inst International Tech RM for Semico, 2y, Experts Nm year Scaling-Factor Norm 45nm 2011 Freq (incl. Drop-Off Heat?) Power More Factors 2nd pessimistic Opinion 8nm now. With this Model we predict the development of Transistors of the next few Years As for Science today, we don’t have a crystal-ball which tells us the future – so we rely on predictions of renomated institutes. One of those is the International Technology Roadmap for Semiconductors ITRS, which is published every two years by experts in the field of semiconductors. On the horizontal axis you see the expected Transistor-Sizes in nm for each Year. The Vertical axis correspondents to the relative Speedup, normalized to 45nm-Architectures. I Plotted the expected Frequency increase in red aswell as the Power-Decrease in Blue. Please bear in Mind that those Roadmap contain way more predictions for different factors – those are just two obvious ones. Since it’s alway’s good to have a second opinion the also resorted to a more pessimistic Prediction by shekar Borkar and Intel-Employee which they labaled Conservative Scaling. So this model tells us about future transistors Year / Estimated Transistor-Size Year / Estimated Transistor-Size Dark Silicon and the End of Multicore Scaling

Core Model Multicore Device Core Power vs. Performance Area vs. Performance Why take this one… …if I could use less power AND Have more Performance? Why take this one… …if I could use The same Area AND have more Performance? Optimal # of Cores Multicore Speed-Up % of Dark Silicone Power vs. Area -> seperatly Standard Performance Evaluation Corporation Right here we see scatter-Plotts of some Processors. To the left we see Data-Points for Core-Power and Performance wich is benchmarked by the Standart Performance Evaluation Corporation. The goal is now to define a Curve, a Pareto-Frontier which defines the optimal Tradeoff between the two. This is achieved by fitting a curve (in this case a cubic-Polynomial) through a selected set of points. To decide which point represents a good tradeoff I show you an example: … To the right we see the same procedure, this time with Area and Performance, where it’s really obvious. We want maximal Performance with minimal Area SPEC = Standard Pervormance Evaluation Corporation Dark Silicon and the End of Multicore Scaling

Device Core Multicore Device Core ITRS scaling Conservative scaling Optimal # of Cores Multicore Speed-Up % of Dark Silicone Combine Plug in [Click] Move down More Perf w. less Power If we now Plug in the Data obtained by the Previous Model (where we looked at the development of Transistors), we can see how those Pareto-Frontiers, this optimal Tradeoffs start to shift downwards as Transistors get smaller. Meaning were getting more Performance with less Power. Dark Silicon and the End of Multicore Scaling

Multicore Model Chip Organisations: CPU vs. GPU 4 microarchitectural features: Multicore Device Core Symmetric Multicore Dynamic Multicore S S P P P P P P P P P P Optimal # of Cores Multicore Speed-Up % of Dark Silicone Constelation of Cores [Click] CPU or GPU Symmertric: Communist: Capabilities, Area, Power Asymmetric: Monolythic Dynamic: Variation of Asymmetric Composed In the last Model we look even a level higher, Here we differentiate between CPU or GPU. Also we consider 4 different Microarchitectual Features. Symmetric Multicore is like the Communistic Approach: Every Core is the same. Each core get’s the same Power-Budget and the same Area. They all operate at the same frequency. Asymmetric Multicore: You have one Big powerfull Monolythic Core optimized for handling serial Parts of Code. You still have multiple smaller Cores at your disposal to help with parallel Workloads Dynamic Multicore is a variation of the Asymetric Multricore, where you turn the big Core off during Parallel Parts and the small cores for serial parts vice-versa. Composed Multicore consists of many small cores if you have parallel Applications, as soon as you need serial power – the can fuse logically together to form a big serial core. Asymmetric Multicore S P Composed Multicore P S Dark Silicon and the End of Multicore Scaling

Device Core Multicore All points along the area/performance Pareto-frontier are considered Adding cores Speed-up is computed As the area or power-limit is hit, we obtain the optimal Number of cores and its speed-up Dark Silicon = ChipArea – UsedArea Multicore Device Core Optimal # of Cores Multicore Speed-Up % of Dark Silicone Optimal Tradeoff Area-Performance Single Core – adding 1by1 Resulting Speed-Up Performance Degredation Dark Silicon = Total Area – Used-Area Now to fuse the previous two Models together with the Multicore Model, we consider all points along the Area-Performance Pareto Frontier. As you remember, we defined the Optimal trade-off between Area/Performance with those curves for each Technology-Generation to come. Then we start adding cores and turn our Single-Core-Machine into a Multicore-Machine. For each core added, we compute the resulting Speed-Up. We keep adding cores until we hit an Area-Limit or a Power-Limit. If we’d keep adding cores we woul’d experience performance degredation. So we know that this must be the optimal Nr of Cores and this is the maximum SpeedUp we can get. If we now wan’t to know the amount of Dark Silicon, we simply Subtract the used Area from the overall Chip-Area. So now we have the Predictions for Optimal # of Cores, maximum SpeedUp and minimum Percentage of Dark Silicon for the future Transistors predicted by the ITRS or the Conservative-Scaling respectively. Dark Silicon and the End of Multicore Scaling

I. II. III. OUTLINE THE PROBLEM/ NOVELTY THE MODEL THE RESULTS
THE GOOD / THE IMPROVABLE Now, let’s take a look at the Results TAKE-AWAYS / THE FUTURE QUESTIONS / DISCUSSION Dark Silicon and the End of Multicore Scaling

THE MODEL Multicore Device Core } Scaling-Models Let’s see what these Models predict { Predictions Optimal # of Cores Multicore Speed-Up % of Dark Silicon } Dark Silicon and the End of Multicore Scaling Device Scaling Core Scaling Multicore Scaling

RESULTS: # OF CORES Using Optimistic ITRS – scaling Tested with PARSEC No explosive growth in core- count The GPU-core-count low due to PARSEC For raytracing-application: Transistor-size: 8 nm Core count: 4864 You Are Here! Many Diagramms [Click] Optimistic ITRS [Click] PARSEC [Click] Decreasing nm + Arrow Blue: CPU Red: GPU [Click] Few GPU cores? GPU not Typical Workload [Click] Ray-Tracing So here we see the expected Nr of Cores for CPU in Blue and GPU in Red. This Data was generated based on ITRS-Scaling wich represents an Optimistic Scaling. Then PARSEC, a benchmark where you can model Applications with variing percentage of Parallelism. So what you see is the Geometric Mean of the Optimal Nr Of Cores over all Test-Run’s over all possible Architecture-Configurations of the Model And as you see there is no Explosive Growth in Terms of Core-Numbers, neither in the CPU or the GPU branch. Right now you might be wondering, why the GPU-Core-Count is that low, and even surpassed. The Paper argues that this is in Part due to the PARSEC Benchmark which wasn’t optimal for modelling GPU-Heavy-Workloads. They made other Tests, using Raytracing (…). Here they predict, that as soon we hit 8 nm we’ll have almost 5000 Cores with only 8% Dark Silicon, wich doesn’t sound to bad. Dark Silicon and the End of Multicore Scaling

RESULTS: SPEED-UP Normalized speed-up, to quadcore Nehalem 45 nm Speed-up in geomean saturates Same Graphic for SpeedUp GM left Pillar, Max right Normalized Even thou max – GM saturates Here you see the Normalized SpeedUp. Even though the best test’s still seem to grow exponentially for CPU -, the Geometric Mean doesn’t look very promising, neither for CPU or GPU You Are Here! Dark Silicon and the End of Multicore Scaling

RESULTS: % OF DARK SILICON
At 8nm: CPU > 50% GPU > 90% !!! GPU Raytracing 8 nm Dark silicon: 8 % You Are Here! CPU of 50% at 8 nm GPU over 90%!!! 8nm Use “Dark” parts for Helper-Threads Example: Data-Compression // Assist GPU threads Onur! This Plot draw’s a really dark Picture. As you see, at 8 nm, half of a CPU should be Dark Silicon. And in GPU’s you could think we aren’t using any Silicon at all. Dark Silicon and the End of Multicore Scaling

The Source of Dark Silicon
SpeedUp vs. Power SpeedUp vs. Parallelism 8 nm around now Horizontal: Power Vertical Normalized SpeedUp to 45 nm PARSEC variing Parallellism All benchmarks saturate Most don’t exceed x10 Parallelism 75 – 100 SpeedUp again to 45nm Power Fixed Point out saturation To see where Dark Silicon comes from we can look at those two Plots. Those where created using multiple different Benchmarks on an 8nm CPU Architecture using the ITRS predictions On the Left you see how SpeedUp increases in an Logarithmic manner as we increase the Power Budget for the Cores. The logarithmic behaviour is explained due to the increase of heat, which limits the amount of silicon we can turn on. On the other hand to the right where we compare speedup as we increase the Parallelism of our code – we see a steep exponential increase from 85% on. This is telling us that we can use more of the available Silicon by parallelizing the code more, if possible. Dark Silicon and the End of Multicore Scaling

MOORE vs. reality ITRS scaling Conservative scaling Dark Silicon Here you see the in Red Predicted the typical upper Bound fo SpeedUp for both ITRS and Conservative Scaling as transistor-Size decreases. In green for comparison you have Moore’s Law. As you see, especially in the case of conservative Scaling, there is quite a gap arising, as soon as we go beyon 16 nm. The paper makes strong implications that this Gap is due to dark Silicon. Dark Silicon and the End of Multicore Scaling

CONCLUSION Problem: Dennard- and multicore-scaling don’t work anymore Dark Silicon is taking over Goal: Find the source of dark silicon Predict the development of core-numbers, performance and dark silicon Method: Model future chip-designs Calculate future dark silicon from model Result: Predictions toward’s % of dark silicon, # of cores and speedUp Power and multicore-scaling is limited by dark silicon Limited parallelism is the primary source of dark silicon Goal Result Parellel Code!!! Dark Silicon and the End of Multicore Scaling

THE GOOD / THE IMPROVABLE THE GOOD / THE IMPROVABLE TAKE-AWAYS / THE FUTURE QUESTIONS / DISCUSSION Dark Silicon and the End of Multicore Scaling

The Good Big impact on the industry Shows where the problems are and how they will develop Covers various architectures and applications Well structured paper Unveiled Dark Silicon and it’s effects Limited Parallelism …but not all Layered This paper hat a big impact on the industriy, as it opend the eyes towards the problem of Dark Silicon and that Multicore-Scaling is no Match for it It show’s where the problems are – the underutilisation due to limited parallelism It covers a wide Array of Architectures and Applications Last but not least: Ist well structured, goes layer by layer deeper into the topic increasing complexity gradualy Dark Silicon and the End of Multicore Scaling

The Improvable Number-of-cores for GPU is a somehow sparse prediction ARM was not considered in this model Calculation of Dark Silicon percentage is explained in only one sentence Is Dark Silicon the sole enemy of Moore’s Law? More information to replicate test-environment How would it behave in an multi-application-environment? For the GPU’s i feel like they where kinda neglected a bit – for example the calculations of the nr. of cores where not really what one would expect, and the tried to fix it up with one ray-tracing example – but it’s not really furfilling – maybe the importance of GPU’s was not that present in 2011 Now this is a Rathole. But is ParSec representative for a real PC? On a complex Systems you have load’s of background services that might run on this darker areas, at least as long as there are no Power/Thermal constraints ARM was not considered due to lack of data – but ARM has gotten extremely important nowadays with the mobile market The main Buzzword, the Calculation of dark silicon is only explained in one brief sentence. Although it might be correct one also might expect a little more ellaboration on this one I felt it implicated that Dark Silicon is the only enemy of Moores Law, without considering other factors that might be around. And this is more of a personal one, but ist really hard to replicate the test-enviroment or get an update to check how this held up in 2018. Dark Silicon and the End of Multicore Scaling

THE GOOD / THE IMPROVABLE TAKE-AWAYS / THE FUTURE TAKE-AWAYS / THE FUTURE QUESTIONS / DISCUSSION Dark Silicon and the End of Multicore Scaling

Take-Away’s Moore’s and Dennard’s Law will no longer apply Multicore-scaling might not be the answer for everything Limited parallelism is at least as problematic as power- constraints What should you take away from this paper? I think its important to know that Moores and Dennards Law will no longer apply, and maybe this gave you an idea why. Also you might now see that just increasing Core-Count is not a perfect solution. And last but not least I wan’t you to know, that it’s not only Power which makes it hard to use all transistors on a chip, but also, the limited parallelism of tasks itself. Dark Silicon and the End of Multicore Scaling

BEYOND THE PAPER: 4 HORSEMEN
THE DIMMED Underclock or only use in bursts THE SPECIALICED Build specialized cores, and only turn on the ones we need THE SHRINKING «Simply remove» Dark Silicon physically THE EX-MACHINA Beyond Silicon Beyond Way’s to fight DS Coined 4 HM of DS-Apok Each maybe undisirable, but together To look beyond the paper, wich was published in 2011, ,what are approaches the Industry might persue to conquer dark silicon? We can’t do much to increase Parallelism in Applications, some applications are inherently serial. But maybe we can do something to conquer the power-limitations. There are four suggested approaches – often symbolized as the four horsemen of the Dark-Silicon-Apocalypse – each one not really desirable, but maybe partially necessairy. The 1st on is the shrinking Horsemen. If we see we can’t use the silicon because it is mostly dark – just cut it out of the design and build smaller chips – this doesn’t really fix the problem though, as it only fights symptoms. The 2ndone is the dimed Horsemen, were we convert dark Silicon into dimmed Silicon, either by underclocking it or only using it in burst and let it cool down afterwards. The specialiced Horsemen tries to eredicate dark silicon, by constructing more task-specific sections in the processor and only turning those on that we need. The last one is the Ex-Machina Horsemen – which basicily say’s that the problem might be the medium of silicon and mosfets itself, and that hope might lie in new approches and materials. Graphics & content: Dark Silicon and the End of Multicore Scaling

THE GOOD / THE IMPROVABLE TAKE-AWAYS / THE FUTURE QUESTIONS / DISCUSSION QUESTIONS / DISCUSSION Dark Silicon and the End of Multicore Scaling

X ? Dark Silicon and the End of Multicore Scaling

Discussion X What new technologies come to mind, that might prevent the end of performance-scaling? Philosophical: What if we hit the end? What might be the consequences? What could Neuro-Science tell us? Lucy - Poster Dark Silicon and the End of Multicore Scaling

Thank you and Happy Halloween!
…and Thank you Giray and Geraldo :D Dark Silicon and the End of Multicore Scaling

Beyond MOSFET’s X Silicon-in-insulator Silicon-on-”nothing” Double-gate FETs FinFETs Vertical FETs Vertical replacement gate FET Ballistic FET Tunneling FET CN-FET MESFET GaN MOSFET Superconductive FET Quantum Graphene Nano-Tubes Neuro-Informatics And many more! Show all the FETS Source: Dark Silicon and the End of Multicore Scaling

PARALLEL TO THE HUMAN BRAIN
X 100 trillion synapses Embody an existence proof of highly parallel, mostly dark operation Source: Last sentence Dark Silicon and the End of Multicore Scaling

AMDAHL’S LAW X 100 % Parallell 50 % Parallel 0 % Parallell Optimal # Cores Speedup % Dark Silicon nm (decreasing) Source Graphics and Data: Presented Paper Dark Silicon and the End of Multicore Scaling

Core Model – (DITCHED) Pollacks Rule: ∆𝑃𝑒𝑟𝑓𝑜𝑟𝑚𝑎𝑛𝑐𝑒≅ ∆𝐴𝑟𝑒𝑎 Power is no longer only constrained by area Empirical Data from 152 Processors Deriving Pareto-Frontiers Multicore Device Core Optimal # of Cores Multicore Speed-Up % of Dark Silicone Core build of T [Click] Previous Studies No longer constrained (Voltage, Frequency) How Model core? [Click] 152 Pareto, Area/Perf Power/Perf seperatly Tradeoff explain In the next Model, we Model the Cores consisting of this Transistors. Previous studies usually made use of Pollacks Rule, which rughthly states that Performance increases equally to the square-root of Area-Increase. But reality show’s that this is insufficient since Performance also depends on voltage and frequency – which no longer scale at historical rates. So this Study decouples Power and Area. They collected Empirical Data from 152 Processors and searched for a Optimal Power/Performance –Tradoff-Line – and an Optimal Area/Performance Line. It does this with so’called Pareto-Frontiers which i will explain briefly. Dark Silicon and the End of Multicore Scaling

Work-Arounds X Dark Silicon and the End of Multicore Scaling

…AND THE END OF MULTICORE SCALING

Similar presentations

Presentation on theme: "…AND THE END OF MULTICORE SCALING"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

…AND THE END OF MULTICORE SCALING

Similar presentations

Presentation on theme: "…AND THE END OF MULTICORE SCALING"— Presentation transcript:

Similar presentations

About project

Feedback