EcoMultiCloud
ue

EcoMultiCloud

Hierarchical approach for green workload management in heterogeneous and interconnected data centers


Project funded by Horizon 2020's SME Instrument
Click Here for more information

The explosion of digital content, big data, e-commerce, and Internet traffic is making Data Centers (DCs) one of thefastest-growing consumers of electricity in developedcountries: nowadays, DCs around the world consume 30 billion watts of power annually (~1,2% of worldwide electricity consumption1), equivalent to the annual output of 30 nuclear power plants producing a total of 200 million metric tons of carbon dioxide2. From these numbers we can understand why DCs are the fastest growing category of ICT emissions: their footprint in 2011 accounted for 0.16 GtCO2e or 17% of the total ICT emissions (see Figure 2). With an estimated growth rate of 7.1%, DCs footprint will reach0.29 GtCO2e in 20203. The 41% of the global amount of DCs are located in USA and the 36% in Europe 4.

datacenter2
img-003

In 2013, American DCs consumed 91 TWh of electricity. This figure is projected to increase to American businesses $13 billion annually in electricity bil aemitting nearly100 millionmetric tons of carbon pollution per year5. In 2013, European DCs consumed 60 TWh ofelectricity. This figure will reach 104 TWh by 2020,costing European businesses €9,6 billionannually in electricity bill6 and emitting nearly 74million metric tons of carbon pollution per year. per year.

The efficient utilization of resources is therefore essential to reduce operational costs, energy consumption,carbon emissions and also to ensure that the Quality of Service (QoS) experienced by users is adequate and adherent to the Service Level Agreements (SLAs).

There are two clear business opportunities that allow to achieve the goal:

  1.  Better efficiency in a single virtualized DC: by dynamically consolidating Virtual Machines (VMs) on the
    minimum number of physical resources, the non-utilized servers can be set to hibernate, hence eliminating
    their respective energy consumption leading to a reduction of carbon emissions and DC operational costs.
  2.  Better efficiency in a multi-site scenario: the dynamic allocation and migration of workload among
    geographically distributed DCs leads additional savings by moving the workload where the energy is
    cheaper/cleaner and/or the cooling costs are lower, according to what is called the “follow the moon” paradigm.

The adoption of interconnected DCs is rapidly increasing: multinational enterprises, financial institutions, telco operators and major cloud providers, are deploying distributed DCs to interconnect both proprietary andthird-parties heterogeneous DCs, in the so-called “Inter-Cloud” scenario.
Inter-DC VM migration is a more novel research topic, as virtualization infrastructures have not offered such features so far. However they will provide such feature in the near future: for example, the vSphere 6.0 release of VMware, launched in February 2015, includes new long-distance live migration capabilities, which will enable VM migrations across remote virtual switches and DCs. While opportunities opened by long distance migrations are huge, involved issues are also extremely complex: among them, we can mention the benefits-to-drawbacks convenience for the workload migrations, the selection of which site from/to migrate, the selection of which specific portion of the workload should be migrated, and the determination of the reassignment of the migrating workload into the target site.

Current approaches aim to solve the problem as a whole, in a centralized fashion, undergoing the risk of originating three main issues: (i) poor scalability, due to the large number of parameters and servers; (ii) poor ability to adapt to changing conditions, as massive migrations of VMs may be required to match a new workloaddistribution strategy; (iii) limitation to the autonomy of the sites, which are often required to share the same strategies and algorithms.

To efficiently address these potential problems, we believe that it is necessary to decentralize part of the intelligence and distribute the decisions points, while still exploiting the centralized architecture and functionalities offered by virtualization infrastructures in single DCs. This naturally leads to a hierarchical infrastructure, in which single DCs manage the local workload autonomously but communicate with each other to route and migrate VMs among them.11

The EcoMultiCloud hierarchical architecture is composed by two layers:

  • the lower layer is used to allocate the workload within single DCs: each site adopts its own strategy to assign VMs internally, with local consolidation algorithms (possibly different from site to site). The lower layer collects information about the state of the local DC, andpasses it to the upper layer.
  • the upper layer is able to exchange information among homogeneous and interconnected sites and drive thedistribution of VMs among the DCs : a set of algorithms – shared by all the sites – are used to evaluate thebehaviour of single sites and distribute the workload among them, both at the time when newapplications/VMs are assigned and workload migration from one site to another is deemed appropriate.
EcoInterCloud arch.1

A key responsibility of the DCM (Data Center Manager) is to analyse detailed data about the local DC and summarize relevant information that is then transmitted to remote DCMs and used for the assignment and redistribution of workload. The nature of the high level information depends on the objectives that must be achieved.

EcoMultiCloud gives to DC administrators the opportunity to achieve the following technical and business goals (validated by HP’s Innovation Center, Cisco Lab and Telecom Italia Labs):

  • Reduction of power consumption and carbon emissions (minimum 30% – maximum 60%)
    The solution consolidates any number of VMs onto the minimum number of physical server hosts, hence delivering autonomic Host Thin Provisioning TM (registered as trademark by the SME Proponent Eco4cloud ) provisioning only the number of physical servers and cores the applications need, rather than the number provisioned. Host Thin Provisioning TM works by switching on and off physical cores as workload varies in real-time. This way, the unoccupiedservers are set to hibernate, hence eliminating their energy consumption. It is important to underline that an idle server consumes more than 50% of the energy consumed when fully utilized. By maximizing the utilization of active servers in the interconnected DCs, CapEx as well as OpEx(in the form of energy savings) gets immediately reduced. Furthermore, by reducing the energy consumption, cooling and sizing gets reduced as well, contributing to indirect additional reductions of OpEx and CapEx, respectively. Our consolidationprocess have been proven in several deployments such asimg-009 Telecom Italia’s and Herning Kommune (Denmark), and validated by industry leaders such as VMware, Cisco Systems,Hewlett‐Packard and IBM. Results affirm that it help reducing power consumption and carbon emissions in the range 30% – 60% and achieving efficiency figures in excess of 90%. Companies are today strongly encouraged to reduce the amount of carbon.
  • Reduction of energy costs (depending on DCs location)
    The cost of electricity is generally different from site to site and also varies with time, even on a hour-to-hour basis,therefore the overall cost may be reduced by shifting portions of the workload to more convenient sites. EcoMultiCloud can bidirectionally shift workloads between facilities depending on a variety of policy‐set criteria, such as to max out the utilization of available capacity, to take advantage of the price and/or availability of electricity at different sites, or dynamically price‐brokering multiple public‐cloud providers as workloads can also be shifted between sites at different times of the day or night (a process also known as “follow the moon”) based on related cost and/or convenience.
  • Quality of Service Management and load balancing among different sites
    Organizations managing multiple DCs want to ensure workloads are proportionally balanced across sites in order to improve the responsiveness of specific sites and reduce the impact on physical infrastructure such as power and cooling systems. In fact the workload must be distributed without overloading any individual site. This may affect the Quality of the Service, which may still be improved by intelligently combining applications (i.e., VMs) having complementary characteristics (e.g., CPU‐bound with RAM‐bound applications). In an inter-site environment, EcoMultiCloud is able to properly balance the load distributed across different sites, as a better balance may help improve the responsiveness of the individual sites, decrease the impact on physical infrastructure, and help prevent overload conditions affecting the QoS.
  • Compliance with Service Level Agreement
    Thanks to the insights and real-time monitoring analytics of critical system parameters provided by EcoMultiCloud, DC Managers can proactively/predictively prevent SLA violations, mitigate risks and increase overall DC reliability positioning the enterprise with a strong Green/Environmental reputation.
  • Reduced Inter-DC Communications
    EcoMultiCloud architecture will also be functional to intelligently determining which workloads should remain within a specific facility rather than being migrated to another site, as in fact not all workloads are equally portable, especially those associated with large data sets. So, in some cases it is more efficient to assign a VM to the local DC instead of moving it to a remote DC depending on many factors, among which the amount of data used by the VM, the available inter‐DC bandwidth and the type of applications hosted by the VMs. For example, choosing a local DC is more convenient if the VM hosts a database server as opposed to a web service just because of the intrinsic characteristics of the respective VMs. All the above mentioned goals are important, yet different DCs may focus on different aspects, depending on the specific operating conditions and on the priorities prescribed by the management, and EcoMultiCloud enables the intelligent combination of choice delivering efficiency and flexibility combined. In fact, the efficiencies and optimizations which EcoMultiCloud introduces to an inter-site environment deliver value on multiple fronts, effectively being based on a novel and distinct degree of additional architectural intelligence by which inter-cloud implementations can operate. These functional values are further enriched by the inherent and differentiating attributes of the utilized algorithms, i.e.: unlimited scalability, technology platform independence, real‐time adaptation, unified management.
img-008
img-009

In order to ensure that  goals of project  are reached, we have carefully analysed the scope and duration of all tasks, as well as their interdependency, developing an adjusted work plan that allows us to optimize time without compromising technical excellence. Special attention has been put on those features identified as critical or having a higher risk either from a technical or from a commercialization point of view. During the first stage of the project (7 months, WP1) the focus will be on the definition of technical requirements and constraints related to the dynamic workload distribution and inter-DCs migrations on which will be performed a basic re-design of the overall hierarchical EcoMultiCloud architecture. The output will be a set of metrics and procedures to assess the capabilities of the devised solution, linked to the business/technical requirements.

In WP2, all data acquired in WP1 will be processed to define and implement LM and DCM modules together with their LM/DCM communication protocol, with the aim to produce a preliminary assessment of the different policies for workload consolidation. Finally, we will proceed with SW integration and related tests.

In WP3, after the definition of a multi-DC environment matching the constraints defined in WP1, data coming from the selected environment will be collected and processed to build a use-case scenario and assess a model for the simulation. During the simulation phase, the impact of the real execution of the EcoMultiCloud algorithms will be analysed and the outcomes will be shared with both the DCs administrators and managers. After a focused tuning and algorithms optimization, the EcoMultiCloud alpha version will be released .

In WP4, the pilot test will be executed: the EcoMultiCloud alpha version will run on Telecom Italia DCs. The output of WP4 will the release of EcoMultiCloud final (beta) version. In parallel way, a Business Innovation Plan incorporating a detailed commercialisation strategy (dissemination, exploitation and IPRs management) and a financing plan in view of market launch, will be periodically revised within WP5.

Finally, WP6 has been devoted o:

  • monitor project progresses and use of the resources allocated;
  • establish an efficient decision making procedure with the aim to identify potential deviations or anticipate
    risks, and therefore to define the proper corrective action;

Eco4Cloud will coordinate and directly carry out or supervise all tasks within the project, both technical and business development activities. Only in specific tasks in which we lack the necessary resources to ensure the best results, we will seek the assistance of subcontractors for expert support

SUB-CONTRACTOR INVOLVED IN THE PROJECT

  • LABOR Industrial Research Lab   

LABOR is a private industrial research and engineering laboratory providing engineering, consulting and technology development specifically targeted at SMEs operating in the EU, in order to boost their growth at a European level through technological innovation. The Engineering and Development team consists of electronic and mechanical engineers, computer scientists, laboratory technicians and industrial chemists. The qualified staff of the company is supported by a network of technology partners and centers of excellence at European level. Labor has been certified ISO 9001 since August 2002 and operates in the Technological Park called Tecnopolo Tiburtino in Rome (Italy). Apart from third party engineering, Labor develops its own technological platforms, for selling or licensing purposes.

The infrastructures include:

  • a laboratory for electronics and automation;
  • an area for innovative prototypes fabrication, development , engineering and testing;
  • a laboratory specialised in industrial chemistry.

In the ICT area, Labor is able to take care of the full development cycle starting from gathering and analyzing requirements, designing, developing and testing software products. This expertise, coupled with proven simulation and data analysis competencies, may cover an essential role in helping the Eco4Cloud team to follow customer oriented and technical effective guidelines during the developing phase.

img-096

Relevant expertise that will contribute to EcoMultiCloud project

  Key personnel involved

 Francesco Costa – A results-driven, customer-focused, articulate and analytical Senior Software Engineer who can think “out of the box”. Very large experience in design and integration problem solving skills. Expert inC/C++,Java, C#, .NET, PHP, and SQL with database analysis and design, applied to Win32/64, Linux32/64, ARM v7, Cortex-M3/4, PIC18/24/32. Skilled in developing requirements specifications, user documentation, architectural systems and artificial intelligence research. He works in Labor since 2010. He has worked in several multidisciplinary projects taking care of different ICT related aspects. Recently he designed the DSS Engine of VINTAGE “A user friendly Decision Support System for an integrated vineyard management, for addressing quality and quantity production variability optimising the use of resources” FP7-GA-286608. Some of the published work was about new challenges in integrating and extending DSS and Data Mining advanced techniques to spatial data of Geographic Information Systems: “GIS, mechanistic modelling and ontology: a performing mix for precision and sustainable viticulture.“ (B. Bois, A. Volta, A. Caffarra, F. Costa, M. Rega, G. Antolini, F. Tomei, S. Galizia, J. Nascimben, D. Crestini, F. Baret, M. Neri, C. Bertozzi, G. Lughi, M. Roffilli, L. Bottarelli, B. Bauer-Marschallinger, S. Hasenauer, S. Campagnolo, D. Delavalle, F. Brossaud, V. Grosso, V. Marletto – Proceedings of the 37th World Congress of Vine and Wine in Mendoza – Nov. 2014) and “INTEGRATION OF REMOTE SENSING IN A DECISION SUPPORT SYSTEM FOR VINEYARD MANAGEMENT” (F. Baret, B. Bauer-Marschallinger, S. Campagnolo, F. Costa, V. Marletto – ESA living planet symposyum – Nov. 2013)

tab 4

The following figure shows the Gantt chart of EcoMultiCloud project. The project was started on 01/12/2015 and will end on 30/11/2017

imgpsh_fullsize

During the first stage of the project (WP1) the focus will be on the definition of technical requirements and constraints related to the dynamic workload distribution and inter-DCs migrations on which will be performed a basic re-design of the overall hierarchical EcoMultiCloud architecture. The output will be a set of metrics and procedures to assess the capabilities of the devised solution, linked to the business/technical requirements.

In WP2, all data acquired in WP1 will be processed to define and implement LM and DCM modules together with their LM/DCM communication protocol, with the aim to produce a preliminary assessment of the different policies for workload consolidation. Finally, we will proceed with SW integration and related tests.

In WP3, after the definition of a multi-DC environment matching the constraints defined in WP1, data coming from the selected environment will be collected and processed to build a use-case scenario and assess a model for the simulation. During the simulation phase, the impact of the real execution of the EcoMultiCloud algorithms will be analysed and the outcomes will be shared with both the DCs administrators and managers. After a focused tuning and algorithms optimization, the EcoMultiCloud alpha version will be released.

In WP4, the pilot test will be executed: the EcoMultiCloud alpha version will run on Telecom Italia DCs. The output of WP4 will the release of EcoMultiCloud final (beta) version.

In parallel way, a Business Innovation Plan incorporating a detailed commercialisation strategy (dissemination, exploitation and IPRs management) and a financing plan in view of market launch, will be periodically revised within WP5.

Finally, WP6 will be devoted to:

  •  monitor project progresses and use of the resources allocated;
  • establish an efficient decision making procedure with the aim to identify potential deviations or anticipate risks, and therefore to define the proper corrective action;
  • guarantee a prompt communication with the EC and other third parties linked to the project;

Work Package 1 – Solution requirements and architecture specifications

This WP has focused on the definition of technical requirements and constraints related to the dynamic management of the workload across a multi-data center architecture. In particular, it had the objective of completing the -design of the overall hierarchical EcoMultiCloud architecture and investigating the exploitation of inter-DCs migrations to efficiently manage the workload. The output of this WP was: (i) the detailed specification of a two-layer architecture of the EcoMultiCloud solution and (ii) the definition of a set of metrics and procedures to assess the capabilities of the devised solution, linked to the business/technical requirements.

This WP is divided into 4 tasks:

id Title Status
1.1 Analysis of relevant business scenarios and objectives in distributed data centres Complete Details
1.2 Requirements Elicitation and Functional Design Complete Details
1.3 Distributed Architecture Specification Complete Details
1.4 Validation Metrics and Plan Complete Details

Deliverable produced for this wp:

id Due date Actual delivery Status
D1.1 31/03/2016 31/03/2016 Complete and submitted
D1.2 31/05/2016 08/06/2016 Complete and submitted. Slight delay experienced (PO was informed)
D1.3 31/05/2016 08/06/2016 Complete and submitted. Slight delay experienced (PO was informed)

Work Package 2 -Data Center Manager (DCM) and Local Manager (LM) Modules.

The main objective of this work package is to exploit the results of WP1 – specifically the design of the hierarchical architecture of the EcoMultiCloud solution and the definition of related requirements and metrics – in order to implement the main modules of the system.  

Specifically, WP2 includes the definition of the LM (Local Manager) and DCM (Data Center Manager) modules, which are the main components of the lower and the upper layer of the solution. Furthermore, WP2 includes the specification and implementation of the communication protocols that are used to exchange data and information between the LM and the DCM, and therefore between the two layers.

The objectives mentioned so far have been achieved during the RP1 (M1 – M12) and they are illustrated in the Deliverables D2.1 and D2.2. The further objectives of this WP are: (i) produce a preliminary assessment of the different policies for workload consolidation and (ii) proceed to the software integration and related tests. These objectives are related to tasks (T2.4 and T2.5) that started in RP1 and will end in RP2.

The WP is in progress, in accordance to the schedule. Specifically, tasks T2.1, T2.2 and T2.3 are completed and they produced the Deliverables D2.1 and D2.2. Such tasks focused on the implementation of the main modules of the upper and lower layer and on the communication protocols adopted to exchange data among such modules. Tasks T2.4 and T2.5 started in RP1 and will end in RP2: the related results will be reported in the Deliverables D2.3 and D2.4, regarding the software integration and the preliminary analysis of the different policies for workload management and redistribution. 

 

This WP is divided into  5 tasks:

id Title Status
2.1 Implementation of the LM Data Model and Monitoring Features Complete Details
2.2 Implementation of the DCM Data Models and Alalysis Features Complete Details
2.3 Implementation of the LM and DCM Cominucaiuon protocols Complete Details
2.4 Preliminary Analysis of the Policies For Workload and Redistribution In Progress Details
2.5 Software Integration and Testing In Progress Details

Deliverable produced for this wp:

id Due date Actual delivery Status
D2.1 30/09/2016 Complete and submitted
D2.2 31/10/2016 Complete and submitted

Work Package 5 -Commercialisation and communication

The main objective of this work package is to exploit the products of other WPs in order to give financial sustainability to the project and generate revenues after the generally availability of the products developed.
Specifically, this WP already generated the result of finding the target market for EcoMultiCloud, segmented by verticals and geographies and consisting of Large Enterprises in the verticals of Telecommunications, Finance and Cloud Providers, in United States and United Kingdom. This niche market consists of 650 Large Enterprises, and could be broadened in a later stage. The market has been continuously monitored and new commercialisation opportunities have been included during the project and described in D5.2, mainly the availability of an additional use case for EcoMultiCloud, without massive intervention on the project from a technical point of view, the Hybrid Cloud.

This WP is divided into  3 tasks:

id Title Status
5.1 Customer development activities Complete Details
5.2 Continuous Business Innovation Plan revision and IPR review In Progress Details
5.3 Commercialisation plan and Communication In Progress Details

Deliverable produced for this wp:

id Due date Actual delivery Status
D5.1 31/07/2016 28/07/2016 Complete and submitted
D5.2 30/11/2016 30/11/2016 Complete and submitted

Work Package 6 -Project Management

The following objectives have been defined for the coordination activity planned for EcoMultiCloud project:

  1. To review and assess the technical excellence of all work carried out and ensure that the deadlines defined are accomplished, while optimising internal resources within the project;
  2. To monitor timing and excellence of all subcontracted tasks, as well as coordinate the subcontractors’ work;
  3. Manage IP knowledge and ensure that aspects of the EC requirements of communication/reporting are met.

 

This WP is divided into  2 tasks:

id Title Status
6.1 Project Management and Coordination In progress Details
6.2 Project Coordination with the EC In progress Details

No deliverable/milestone is foreseen under this WP.

Publications

Scientific papers and technical reports, listed below, were published during EcoMultiCloud project. They cope with different and important aspects of the workload distribution among  multi-site Data Centers.  For each work, references are provided to the paper and/or the slides of the talk and/or the presented poster.

For more information please contact the Scientific Director of the company, Carlo Mastroianni.

A. Forestiero, C. Mastroianni, M. Meo, G. Papuzzo, M. Sheikhalishahi, “Hierarchical Approach for Efficient Workload Management in Geo-distributed Data Centers“. IEEE Transactions on Green Communications and Networking, August 2016. Note: Early Access. URL: http://dx.doi.org/10.1109/TGCN.2016.2603586. ISSN: 2473-2400

Carmine De Napoli, Agostino Forestiero, Demetrio Laganà, Giovanni Lupi, Carlo Mastroianni, Leonardo Spataro, "Business Scenarios for Geographically Distributed Data Centers". Tech. Rep. RT-ICAR-CS-16-03, ICAR-CNR, May 2016 URL: http://www.icar.cnr.it/tr/2016/03
Carmine De Napoli, Agostino Forestiero, Demetrio Laganà, Giovanni Lupi, Carlo Mastroianni, Leonardo Spataro, "Efficiency and Green Metrics for Distributed Data Centers". Tech. Rep. RT-ICAR-CS-16-04, ICAR-CNR, September 2016 URL: http://www.icar.cnr.it/tr/2016/04
horizon-2020-e-eu

This project has received funding from the European Union’s Horizon 2020 research and innovation programme, under Grant Agreement No. 712363