Presentation on theme: "Request Dispatching for Cheap Energy Prices in Cloud Data Centers"— Presentation transcript:
1 Request Dispatching for Cheap Energy Prices in Cloud Data Centers Hiroshi Yamada (TUAT) In cooperation with Takumi Sakamoto( ), Hikaru Horie, Kenji Kono (Keio Univ.)
2 Cloud Service Providers Make use of geographically-distributed data centersOffer users a large amount of resources in a pay-as-you-go mannerEnable us to easily use resources as needede.g. Amazon has more than tens of DCs over the worldCloud platforms are very common. The current could service providers, like Amazon, Google, Microsoft, commonly make use of geographically-distributed data centers. They orchestrate these data centers to provide users a large amount of resources. Cloud users can utilize them in a pay-as-you-go manner, which means they can easily use resources as needed.For example, Amazon has more than tens of DCs over the world. It has DCs in London , Tokyo, US Dallas, Seattle and so on.If a cloud user needｓ more resources, he or she can use these DCs’ resources to provide the services.TokyoLondonDallasMiamiAmsterdamSeattle
3 Energy cost is EXTREMELY high Millions of dollars of electric costs per year [Qureshi+ SIGCOMM ’04]Google spends $38 million on electricityOther large companies pay tens of million dollarsSignificant financial overhead in the management costEnergy prices can be higher and higherLimited fossil fuelRestriction to build nuclear power plants⇒ Lead to higher charges for the use of cloudsIn the management of these DCs, energy cost is a significant concern. It is extremely high recently.Some document reports that energy costs are equal to millions of dollars per year.For example, google annually spends 38 million dollars for electricity. Additionally, other large companies pay tens of million dollars for electricity.This means that energy cost is significant financial overhead in the management cost.Unfortunately, energy prices tend to become more and more expensive due to limited fossil fuel and restriction to build nuclear power plants.This situation is more severe in Japan.This fact can lead to higher charges for the use of clouds so we want to avoid this situation.
4 Existing solution for reducing energy costs Forwards client requests to DCs in cheaper energy price regions [Qureshi+ SIGCOMM ’09]Average utilization in DCs is 20〜30%Servers can be replicated in different DCsThe previous work has tackled this problem. To reduce energy cost, the previous work dispatches client requests to DCs in cheaper energy price regions. The insight of this work is to pay attention to the fact that energy prices are different in different places. And, the average utilization in DCs is 20 to 30 % for successfully handling request bursts, and server instances can be easily replicated among DCs like Amazon EC2.This graph shows energy prices change over time in the US. This shows three places’ energy price, California, Midwest, and New York.X-axis is time, and Y-axis is electric price.We can see that electric priceｓ change over time, and the price is different in each place.Energy prices are fluctuated over time in the US.
5 Challenges for dispatching client requests in clouds DCs’ capacitiesThe previous work dispatches requests only to the DC in the cheapest energy price regionEven if the DC is overloaded⇒ Service qualities can be degradedLose 1 % income if the response time is 10 % increased in Amazon [Kohavi+ ’07]Various factors to be consideredDC loads, response time, ・・・⇒ Cannot dispatch requests based on only energy pricesHowever, the existing solution cannot be applied to clouds. There are two challenges in energy-price-driven request dispatching.First is DCs’ capacities. The existing solution just consider energy price, so it dispatches requests to only the DC in the cheapest energy price region even if the DC is overloaded, which means the services running on the DC are severely degraded. And the DC cannot handle suddenly-occurred request burst.This is critical for services. For example, if response time is 10 % increased in Amazon, income is 1 % decreased.Second is to consider various factors in dispatching requests in clouds. For example, such as DC loads, response time, and so on.Since the existing work takes into account only energy prices, these demands are not satisfied.
6 ProposalDispatching client requests to DCs in cheaper energy price regions, considering:DCs’ capacitiesVarious factors such as DC loads and response timeAssumptionsKnow the real-time energy prices that vary hourlyUse a function to automatically launch servers inside DCs like Amazon auto scaling functionRequestsDispatcherTo address this issue, we presens an approach to dispatching client requests to DCs in cheaper energy price regions, considering DCs’ capacities and additional factors such as DC loads and response time.Our dispatcher receives client requests, and dispatches them to DCs in cheaper energy price regions, satisfying predefined demands such as DC loads.We have the same assumptions with Qureshi’s SIGCOMM paper.First we know the real-time energy prices that vary hourly.Second, we use a function to automatically launch servers inside DCs like Amazon auto-scaling function.$1 / MWh$2 / MWh$3 / MWh
7 ContributionsPropose a mechanism to dispatch requests to save energy prices, satisfying pre-defined demandsShow an example algorithm that satisfies pre-defined demands and can be integrated to our dispatcherImplement our prototype and perform simulation evaluation using real dataOur contributions are here.We propose a mechanism to dispatch requests to save energy prices with pre-defined demands satisfied.We also show an simple algorithm that satisfies pre-defined demand, DC loads and response time, and can be integrated to our dispatcher.Finally, we implement our prototype and evaluate it in simulation using realistic data.
8 Design of request dispatcher RequirementsScalabilityHas to handle a huge amount of requests sent to DCsAbility to follow frequent-redirection-changesEnergy price and DC loads are changed frequentlyAvailabilityEasy to fail to redirect requests if the dispatcher is fragileSolution: Mapping NodesRun between clients and DCs in a P2P fashionSatisfy the three requirementsTo design our dispatcher, we have to consider three requirements. First is scalability. The dispatcher has to successfully handle a huge amount of requests sent to DCs.Second, the dispatcher needs to follow frequent-redirection-changes since energy price and DC loads are changed frequently. So, it needs to have an ability to do it.Third is availability. If the dispatcher is fragile, client request redirection is easily failed.To achieve the requirements, we prepare mapping nodes that run between clients and DCs in a P2P fashion.
9 Mapping Node Constructs a distributed hash table (DHT) Key: a host name, Value: Mapping EntryMapping entry contains the host’s IP and RTT (for cache)Returns IP address like DNS serversClients use the returned address to connectMapping Node in details.Mapping nodes work together. They construct a distributed hash table (DHT) to achieve the three requirements I said early, availability, scalability and so on. They work like a DNS server for client.Mapping nodes manage a host name as a key, and mapping entry as a value. Mapping entry is a data object that contains host’s IP and RTT for cache in a mapping node. Mapping node returns an IP address to the client when its request is received.First, client sends a request to a known mapping node to get an IP address of the server. Mapping node searches the mapping entry of the given server name.It returns the IP address to the client after finding the mapping entry.
10 A Request Dispatching Algorithm Used for mapping nodes to return IP addressesCalculates dispatching request ratio in each DCA DC to forward is decided based on the ratioCloud service providers can change weightsExample: request dispatching based on DC load, response time, and energy pricesIn this work, our mapping node employs this request dispatching algorithm to decide IP addresses to be returned.This is very simple so there are a lot of rooms to improve and can borrow novel algorithms from various previous work. But an important point is that we can develop an algorithm that is applicable to mapping nodes.This algorithm decides percentages of requests to send to a DC, following pre-defined threshold.It considers three factors, DC load, response time, and energy prices. Each factor can be set weight that is adjusted based on which factor is more important.In this algorithm, the weigh changes in a situation. We set these functions to reduce percentages when response time and DC loads are over the threshold.DC loads. Weight is decreased as DC loads are more severeResponse time. Weight is 0 when latency is over 70 msecEnergy prices. Weight is smoothly decreased as the price is higherweight
11 Simulation Evaluation Implement a prototype on Overlay Weaver [Shodo+ ’06]DC emulator: CloudSim [Buyya+ ’09]ParametersEnergy prices data: 1st week in the US on Aug. 2010Workloads: E-commerce DC workload, borrowing from [Heller+ NSDI ’10]Constraints (Threshold)DC loads: less than 80 %Response time: less than 70 msWe implemented our prototype on Overlay weaver that is a DHT framework and evaluate it on a DC emulator named CloudSIm.We also used the following parameters. We used energy price data for the first week in the US on August 2010,and E-commerce DC workload borrowed from Heller’s NDSI paper.In addition, we set thresholds for DC loads and response time to 80% and 70 msecond respectively.For comparison, we prepared 4 request dispatching policies. Random, Latency & price, Latency & load, and our proposal.Random dispatches request randomly, Latency & price consider response time and energy price, Latency & load considers response time and DC loads,And, our proposal consider all.Compared with the following policiesRandomDispatch requests randomlyLatency & PriceConsider response time and energy priceLatency & LoadConsider response time and DC loadsOur ProposalConsider response time, DC loads, and energy price (shown in the previous slide)
12 Result （Energy price and constraint violation） Our proposal lowers energy prices without the constraint violation28 % better in energy prices than Random⇒ Saves several hundred dollars if annual energy cost is several thousand dollarsEnergy price nomalized by RandomConstraint ViolationLower is betterThe results are here.The left graph is energy price, and the right graph is constraint violation which means whether DC loads and response time are over the threshold.We can see that our proposal successfully lowers energy prices without constraint violation. Random and Latency & Price violate the constraints.Specifically, our proposal is 28 % better in energy prices than Random, which means it saves several hundred dollars if annual energy cost is several thousand dollars.Lower is better
13 Result （DC loads） Our proposal successfully regulates DC loads DC loads in Latency & Price are fluctuated severelyThreshold = 80%RandomLoads are changed along with the workload.1234567DayThreshold = 80%Latency & PriceLoads are sharply fluctuated.The results of DC loads are like this.We can see that our proposal almost successfully regulates DC loads under the threshold.In latency & price, the loads are severely fluctuated since it does not consider any DC loads situation.1234567DayThreshold = 80%Latency & Load Loads are changed along with the workload.1234567DayThreshold = 80%Our ProposalLoads are changed along with the workload.1234567Day
14 Related WorkEnergy-price-driven request dispatcher [Qureshi+ SIGCOMM ’09]Dispatches requests to DCs in a cheap energy price regionOnly consider energy priceLoad-balancer for DCs [Wendell+ SIGCOMM ’10]Dispatches requests to DCs based on their loadsDoes not consider energy pricesDHT-based DNS [Ramasubramanian+ SIGCOMM ’04]Achieves fault-tolerant and high performance DNS using DHTNot focus on energy prices and cloud user policyRelated work.Qureshi proposes energy-price-drive request dispatcher.However, as I said, it considers only energy prices so it cannot take into account other factors like response time and DC loads and so on.The Load-balancer for DCs dispatches requests to DCs based on their loads.This balancer does not consider energy prices.DHT-base DNS is proposed and it achieves fault-tolerant and high performance DNS using DHT.We can use this techniques to mapping node to enhance it.
15 ConclusionsProposal: Energy-price-driven request dispatcher for cloudsTakes response time and DC loads into accountLeverages a DHT mechanismOur simulation result shows that our dispatcher saves energy costs without violation of pre-defined constraintsFuture workEnhance policies used in mapping nodesEvaluate our prototype in the real worldConclude this talk.We propose an energy-price-driven request dispatcher for clouds that can take into account various factors like not only energy prices but DC loads, response time and so on.It leverages a DHT mechanism to achieve its scalability and availability.Our simulation result shows that our dispatcher saves energy costs without violation of pre-defined constraints.Future work includes to enhance request dispatching policies used in mapping nodes, and to evaluate our prototype in the real world.
16 Result （Mapping node execution time） LatencyMeanMedian90th %Legacy DNS (original)382 ms39 ms337 msLegacy DNS (revised)246 ms30 ms163 msMapping Node102 ms52 ms204 ms[ms]