Workpackages to achieve the objectives

The activities of the Contrail Project are meticulously defined in a projectplan. Through a project break-down into work packages, the comprehensive task has been made feasible.

WP 1 Project management 
WP 2 Infrastructure as a service federation (IaaS) 
WP 3 Service level agreements (SLA) 
WP 4 Virtual infrastructure network (VIN) 
WP 5 Computational resource management for virtual cluster platforms (VCP) 
WP 6 Global autonomous file system (GAFS)
WP 7 Security in virtual infrastructures 
WP 8 High level services 
WP 9 Runtime environments 
WP 10 System archtitecture 
WP 11 Integration, testing and release management 
WP 12 Applications and use cases 
WP 13 Testbeds 
WP 14 Communication and dissemination 
WP 15 Demonstrators
WP 16 Exploitation and technology transfer

WP 1 Management

Leader

INRIA, France

Tasks

T1.1: Setting-up and maintenance of collaborative tools.

T1.2: Receiving and distributing EC funds to partners according to the preset budget.

T1.3: Monitoring project progress and expenses against the planned schedule and ensuring that milestones are met and deliverables are produced in due time.

T1.4: Devising strategies for managing risks.

T1.5: Producing financial and technical reports in accordance with the contract schedule and on request.

T1.6: Overseeing results exploitation and IPR related matters.

WP 2 IaaS federation

Leader

CNR, Italy

Objectives

The goal of this WP is to develop interfaces and mechanisms which enable assembling a federation of Clouds from heterogeneous hardware resources and Cloud providers. This requires defining and implementing - mechanisms for identity management in federated Clouds

abstractions and mechanisms that enable resource policy control and community management across federated Clouds - resource selection mechanisms at the Cloud federation level exploiting QoP and SLAs.

Tasks

T2.1: Requirement Specification for Federation Management. Task defining the requirements for federation management, surveying CONTRAIL objectives, assessed standards and research results

T2.2: Architecture Design of Federation Management Support. Architecture design of the federation management support, and its revision for the second implementation

T2.3: Federation Management and Identity management. The objective of this task is to develop the technology for managing federation of Clouds, addressing the stacking and interoperation of security layers.

T2.4 Managing Security Policies in IaaS Federations. This task will develop formal strategies for the enforcement of federated security policies, and provide implementation solutions.

T2.5: SLA-Based Resource Provisioning. The purpose of this task is to provide components for resource provisioning in each layer of the CONTRAIL stack according to SLAs, selecting resource providers capable of fulfilling a specified QoS contracts.

T2.6: Cloud Provisioning API. Design and development of an API for Cloud provisioning which incorporates relevant Cloud standard API, and extends them to expose all CONTRAIL federation-level features.

T2.7: Standard Provisioning Interfaces. Providing web-based user interfaces to allow access to and management of the CONTRAIL platform

WP 3 Service level agreements

Leader

HP IIC, Italy

Objectives

The objective of this WP is to develop the technology to manage SLAs in Clouds. The focus will be on providing support for the full life cycle of SLAs, including creation, instantiation and enactment of agreements at all levels of the Cloud services stack: infrastructure, IaaS federations, and platforms as a service. In Contrail, SLAs will specify the Quality of Service (QoS) and the Quality of Protection (QoP) provided by service providers.

Tasks

T3.1: Requirements for SLAs in Clouds. This task will define the requirements needed for SLA management in Contrail service stack. The task will work closely with the application case studies (WP12) and the business model task (T16.3)

T3.2: Definition of Quality of Service and Protection Terms. This task will define QoS and QoP parameters at different levels of Contrail service stack. Performance, availability and security guarantees for infrastructure components will be defined, covering network (in collaboration with task T4.2), compute
(in collaboration with task T5.7) and storage (in collaboration with task T6.5)


T3.3: Architecture for SLA Management Services. The purpose of this task is to develop the specification and architecture of the SLA management services in Contrail. After a first release based on the output of tasks T3.1 and T3.2, this task will revise the specification and architecture according to the experimental results obtained.


T3.4: SLA Management Services. This task will implement the functionalities and components required for autonomous SLA management. This involves a SLA repository to store SLA templates and for SLA-negotiation purposes; SLA instantiator, in charge of instantiating and enacting SLAs; and a SLA evaluator, which receives monitored information and evaluates whether a SLA has been violated.
 
T3.5: Monitoring. The purpose of this task is to develop a component monitoring the adherence to SLA guarantees in the different layers of Contrail service stack.

T3.6: Accounting and Billing. The task will develop components for account and charging for resources used across one or more service providers.
 

WP 4 Virtual infrastructure network (VIN)

Leader

VUA, The Netherlands

Objectives

The goal of the VIN work package is to integrate loose infrastructure components into a cohesive, virtual cluster platform with a both secure and efficient virtual network. A virtual cluster securely integrates both file storage (GAFS) and remote client machines. Authentication is based on Cloud-level user accounts; all communication is encrypted. The VIN serves within a cloud, and extends to other infrastructures (for cloud federation), to the GAFS storage, and to external client machines.

The cloud-local VIN infrastructure establishes a VIN across all nodes of a cloud that belong to the same user account (and node allocation). The infrastructure is based on a self-organizing overlay network, in combination with shortcut data communication within physical local networks.

VIN interconnects will provide different (performance) service levels, based on the guarantees of the underlying, physical networks, and on traffic shaping within the VIN overlay itself.

For cloud federation, the VIN's of each Cloud federate and establish a joint routing overlay, taking care of connectivity restrictions like firewalls, NAT's, and non-routed local networks.

The GAFS file system integrates into a VIN by exporting its volumes to all infrastructure nodes of a cloud (federation), based on the same authentication and encryption mechanisms as for the VIN itself.

Cloud users can communicate with their cloud infrastructure by joining the VIN from their client machines; immediately benefitting from the secure virtual network to and within their Cloud.

Tasks

T4.1: VIN core overlay. Development of the Cloud-local VIN infrastructure

T4.2: VIN SLA. Development of different service levels for VIN interconnects and their implementation. Integration of SLA mechanisms with the computational resource management for virtual cluster platforms.

T4.3: VIN federation. Development of federation mechanisms for multiple, Cloud-local VIN's. Integration of federation mechanisms with the computational resource management for virtual cluster platforms.

T4.4: GAFS-VIN integration. Development of a VIN-aware GAFS client and of corresponding mechanisms on the GAFS server side.

T4.5: VIN client integration. Development of client-side software for secure, remote user access.

WP 5 Computational resource management for virtual cluster platforms (VCP)

Leader

INRIA, France

Objectives

The goal of WP5 work package is to specify, design and implement the functionalities needed to manage computational resources during the whole life cycle of virtual cluster platforms. Computational resources need to be managed in such a way to guarantee the QoS of active virtual cluster platforms and to comply with the efficiency policies decided by the cloud providers.

Tasks

T5.1: Specification of the architecture for virtual cluster platform life cycle management. We will specify the architecture of the computational resource management service and the mechanisms needed to handle virtual cluster platforms. XtreemOS AEM service will be leveraged according to the needs of the CONTRAIL project in order to manage virtual cluster platforms in cloud federations.

T5.2: Discovering and allocating resources to virtual cluster platforms. We will design and implement the resource discovery service allowing to find a node or a set of nodes that can host a given virtual cluster platform, knowing its requirements in terms of computational resources (number of cores, memory size, network characteristics).

T5.3: Exporting VIN and GAFS to guest OS'es. VIN and GAFS need to be exported to guest OS'es of a virtual cluster platform in order to allow isolated communication within a virtual cluster platform and to provide read/write access from a virtual cluster platform to data in stored GAFS. We will develop these exportation mechanisms.

T5.4: Deploying customized operating system images in dynamically provisioned virtual cluster platforms. We will design and implement tools for automatically deploying operating system images in the VM composing a virtual cluster platform.

T5.5: Development of efficient snapshot mechanisms for virtual cluster platform management. We will design and implement efficient snapshot mechanisms for virtual cluster platforms.

T5.6: Managing virtual cluster platforms. We will design and implement features such as VCP migration and VCP images storage in GAFS.

T5.7: Enforcing QoS for virtual cluster platforms while aiming at efficient computational resource management. This task will investigate how to enforce QoS parameters for virtual cluster platforms while providing efficient resource management. It will leverage from virtualization techniques applied in XtreemOS such as Linux containers and OpenVZ in order to achieve fine-grained control of quality parameters such as memory size, network restrictions, etc.
 

WP 6 Global autonomous file system (GAFS)

Leader

ZIB, Germany

Objectives

This work package provides a reliable and distributed file system for users as well as for the cloud infrastructure. The file system offers users a general purpose Storage-as-a-Service that can be used from within the cloud (i.e. the user's VCPs) but also from the outside via the Internet. In addition, the file system will provide the storage necessary for operating the CONTRAIL infrastructure, like storing the customer's VCP images or system logs and transferring VCP snapshots. We will extend and enhance the distributed XtreemFS file system to implement the Global Autonomous File System.

Tasks

T6.1: Core File System Maintenance. Maintenance and enhancement of the file system core which includes the client, the metadata and the storage servers.

T6.2: File and Metadata Replication. The goal is to implement a full file and metadata replication with automatic failover. The existing mechanisms in XtreemFS will be used as a building block.

T6.3: Autonomous Storage System. Design, simulate and evaluate the zero configuration and self-healing features. Find appropriate solutions for implementation in the file system. Provide an interface for monitoring information on the state of the VMs and host machines .

T6.4: Self-Tuning Mechanisms. Investigate techniques to automatically detect performance problems and to resolve and prevent them. The self-tuning mechanisms build upon the autonomous configuration capabilities of the file system.

T6.5: Quality of Service. Research possibilities to implement QoS in a distributed file system and required support from other system components like the operating system and networking infrastructure.  provides the network/VIN part of Quality-of-Service.

T6.6: Alternative Interfaces. Implement non-POSIX interfaces for the file system for special use cases, e.g. a HDFS (Hadoop file system) interface. Validation and testing of the HDFS interface with Hadoop, optimization of the HDFS interface based on use-cases.

WP 7 Security in virtual infrastructures

Leader

STFC, United Kingdom

Objectives

The objective of this work package is to design and implement the set of services that guarantee a secure operational environment of virtual infrastructures in Contrail. Cloud computing brings new challenges from the security point-of-view. On one side, centralisation of data means the risk of insider threats from within the Cloud provider is a significant concern. On the other side, Cloud models as IaaS and PaaS create differing trust boundaries that must be accounted during the development, testing and production deployment of applications.

Tasks

T7.1: Requirements and Models Security in Virtual Infrastructures. This task aims at defining security requirements at the system level, including end-users security requirements and requirements imposed by the underlying infrastructure. In addition, threat models will be developed for the system as whole, as well as for VIN, GAFS and VCP.


T7.2: Security Specification and Architecture. The purpose of this task is to develop the specification and architecture of the security components in Contrail. A first specification and architecture will be developed based on the security requirements defined in T7.1. After the first release, this task will revise the specification and architecture according to the experimental results obtained at that time.

T7.3: Controlling Access in Virtual Infrastructures. This task aims at evaluating and implementing finegrained access control for the core virtual infrastructure. It includes secure access to virtual machines and applying TPM technologies for securing hypervisor and guest OS, providing a distant chain of trust.


T7.4: Security through Compartmentalization and Isolation. This task will develop techniques to guarantee that access to one virtual machine/server does not reveal knowledge about other virtual machines/servers and services running on the shared physical infrastructure.


T7.5: Security Analysis, Test and Evaluation. This task will carry out the necessary analysis, testing and evaluation of the security mechanisms developed in Contrail.
 

WP 8 High level services

Leader

ZIB, Germany

Objectives

The objective of this work package is to design and implement the set of services that guarantee a secure operational environment of virtual infrastructures in Contrail. Cloud computing brings new challenges from the security point-of-view. On one side, centralisation of data means the risk of insider threats from within the Cloud provider is a significant concern. On the other side, Cloud models as IaaS and PaaS create differing trust boundaries that must be accounted during the development, testing and production deployment of applications.

Tasks

T8.1: Geographically distributed key/value store. Scalarix is a distributed key/value store providing transactional access to the stored data. For a large number of scenarios, it provides a scalable alternative to traditional SQL databases. The CONTRAIL use-cases will be used for the evaluation of Scalarix. We will adapt Scalarix for the CONTRAIL infrastructure and investigate an alternative query language to offer applications a scalable alternative to SQL servers.

T8.2: High-throughput elastic structured storage. The main goal of this task is to provide a high-throughput structured storage that can be used for both data analysis using scalable methods such as Map/Reduce as well as service and application storage back-end. For the implementation, we will use an existing storage back-end BabuDBa to build a prototype and optimize it for usage with the GAFS (e.g. replica location awareness). This task involves optimization and testing together with the HDFS interface from WP6.
 
T8.3: SQL Servers. This task will provide automatic set-up and configuration of an SQL server within the cloud. This will alleviate users from manually installing and configuring the SQL server. Moreover, users don't need to spend time to optimize the setup of the SQL server for the underlying infrastructure. This task includes the implementation and the optimization of server set-up based on infrastructure as well as the evaluation and validation of optimizations.
 

WP 9 Runtime environments

Leader

VUA, The Netherlands

Objectives

The work package on Runtime Environments focuses on presenting attractive programming models for helping developers to automatically take advantage of the advanced features of the Contrail platform. Runtime environments automatically offer elasticity, scalability and performance dependability to applications designed according to its design principles.

Tasks   

T9.1 Efficient map/reduce Map/reduce is used for data processing, often on a very large scale. Therefore, an efficient execution of these jobs requires a tight integration of the map/reduce framework with the underlying storage system. For this task, we will analyze and optimize the Hadoop open source framework for the Contrail infrastructure, i.e. the Global Autonomous File System and the highthroughput elastic storage.

T9.2: Scalable service-oriented application hosting. Cloud platforms are very popular to host Web applications and service-oriented applications. We will design and build a platform for scalable hosting of such applications that makes use of the Contrail infrastructure to support this important class of applications [VUA]. This platform is directly comparable to AppEngine and AppScale. The unique features developed within Contrail include innovative resource provisioning algorithms for multi-service applications and the ability to store data using either traditional SQL databases or cloud data services such as the key-value store from task T6.1.

T9.3: Autonomic workflow execution. Large workflows need to execute many compute-intensive jobs, aiming at high throughput and efficient data file staging. In clouds, with various compute offerings, autonomically optimizing workflow execution w.r.t. completion time and cost is as technologically challenging as it is important for a large group of applications from both academia and industry. The goal of this task is to build an autonomically operating execution engine for workflows and other high-throughput computations, like parameter sweeps or bag-of-tasks applications, as they are frequently found both in eScience and in industrial engineering settings. Such an execution engine gets both an anticipated, overall completion time and an available budget as input parameters. It then allocates compute resources for executing individual tasks, while monitoring the overall progress, adapting the number and kind of compute resources during execution, in order to meet time and budget restrictions.

WP 10 System archtitecture

Leader

INRIA, France

Objectives

All involved partners jointly collect requirements (based on input from SP5) and devise an overall architecture of the Contrail system, along with dependencies and interfaces between the software packages produced by SP1, SP2, and SP3, forming the basis for software integration and release preparation within WP11.

Part of the system architecture are criteria and metrics for the evaluation of individual components and their interplay. Such evaluations form the basis for both informed revisions of the overall architecture itself, and for application-driven evaluation in SP5. - WP4.1 will produce reports on the overall system architecture, on software interfaces, and on system evaluations. There will be two iterations of these reports, one for the initial and one for the final design.

Tasks

T10.1: Global architecture specification: major components of the CONTTRAIL system, interactions between these components.

T10.2: Detailed specification of the component interfaces.

T10.3: Component and system evaluation.

WP 11 Integration, testing and release management

Leader

Xlab, Slovenia

Objectives

All involved partners jointly collect requirements (based on input from SP5) and The main objectives of WP11 will be : Provide a coherent development environment for the partners along with best practice coding rules for high quality codes. Package all contributions of all software in a coherent, manageable and releasable set. Test and validate the software packages. Provide a coherent Linux distribution that will integrate all the specific project elements and a full featured Linux distribution.

Tasks

T11.1: Define and enforce best coding practices to ease inter software integration.

T11.2: Manage a common programming environment, version management, bug tracker, and a fast build environment based on Contrail cloud infrastructure.

T11.3: Community support of Contrail: bug tracker, mailing-lists, IRC.

T11.4: Regression tests: provide a well-defined and well-known environment on the internal testbed for early diagnosis of implementation errors, used in repeatable automated regression tests.

T11.5: Integration of Contrail components: implementation, validation and feedback of interfaces in T11.2.

T11.6: Build Contrail software releases integrated in a Linux distribution.
 

WP 12 Applications and use cases

Leader

Tiscali, Italy

Objectives

The goal of this work package is to evaluate, test and demonstrate the technology developed in the core research work packages SP1–SP4 through a set of heterogeneous reference applications.

Tasks

T12.1: Use cases definition and validation methodology. The use cases and their deployment scenarios will be formally defined and described in details. Based on the application and use cases characteristics, requirements will be derived as input for the work packages dealing with architectural design and system integration. Validation methodologies and assessment metrics will be defined and used for test and verification of the project results.
In the later phases of the project, the initial requirements and use cases will be revised, extended and modified where appropriate, driven by the activities in the technical work packages and the experiences gained from the experimental evaluation.

T12.2: Porting of applications. This task includes the porting of the applications to the Contrail platform. This work also covers the suitable modifications of the applications to efficiently use the advanced features and interfaces of the High Level Services and Runtime Environments developed by SP3. Also the applications will be leveraged to suitably demonstrate the cloud management (SP1) and security capabilities provided by the project.

T12.3: Experimental evaluation. This task will commence with the definition of suitable test benchmarks and test frameworks. Ported applications (T12.2) along with test frameworks will then be used to perform an experimental evaluation of models, tools, and platforms provided by the technical work packages. Evaluation criteria are defined by test benchmarks providing metrics for, e.g., latency, throughput, usability, development productivity, or adaptive system behavior. Experiments may evaluate Contrail releases as integrated systems or they may focus on the various Contrail features like map-reduce and web server runtime environments, or on high level services supporting key-value stores, XOSAGA and SQL.

WP 13 Testbeds

Leader

INRIA, France

Objectives

WP13 aims at managing testbeds for the Contrail project.

Internal testbed: this testbed connecting compute nodes provided by some partners is used for software component integration and testing.

Demonstration testbed: this permanent testbed will run a few selected use cases and will be open to external users through the FIRE portal.

Validation testbed: the validation testbed will be used for large scale validation of the Contrail software stack. The resources for this testbed will be provided by some Contrail partners (French Aladdin/Grid'5000 large-scale distributed platform, Dutch DAS and the German D-Grid), whenever possible through the FIRE portal, or hired from cloud IaaS providers.

Tasks

T13.1: Internal testbed management. Design, installation and updating the internal testbed that we will use for component regression tests.

T13.2: Demonstration testbed management.

T13.3: Validation testbed.

WP 14 Communication and dissemination

Leader

Genias, The Netherlands

Objectives

This work package focuses on communication, dissemination, training and technology transfer. Training activities will be both internal and external. The main objectives of this WP are thus:

Collaboration with ICT IoS Projects.

Dissemination of the project results to targeted audiences.

Internal and external training.

Tasks

T14.1: Collaboration with ICT IoS Projects. The collaboration task covers the liaison and co-operation activities with other ICT projects under the WP2009/2010 Objective "Internet of Services, Software and Virtualisation". The cooperation aims at exploiting synergies between the projects and increasing the impact of the ICT initiative.

T14.2: Dissemination of project results. A dissemination plan will be produced and regularly updated that will contain a list of the dissemination activities of the previous periods, and a plan for the coming period. The plan will contain a dissemination matrix with the targeted audiences, the dissemination tools and the dissemination goals. The target audiences include the scientific community, peer developer community, potential users, general public. The tools used will include a project web site, flyers, background documentation, press releases. Also modern tools like YouTube and blogs will be used.
 
T14.3: Organization of training activities will start with training sessions for researchers working on the project. Later this will be expanded to potential users of the system. For each training session material will be produced and made available on the project website.

WP 15 Demonstrators

Leader

Tiscali, Italy

Objectives

The goal of this WP is to further expand some of the applications developed in WP12 transforming them into large scale, market oriented demonstrators fully open to final users. Many aspects of the Contrail architecture and technology will be tested under heavy load conditions on a real production scenario. Information on performance, scalability, security and usability issues will be collected and reported so that corrective actions and fine tuning of the platform can be accomplished. At the end of the project, the demonstrators will become full featured and almost market-ready applications that can be easily transformed in commercial products by the industrial partners after the end of the Contrail project.

Tasks

T151: Specification and validation methodologies.

T15.2: Early integration and testing.
 
T15.3: Final demonstrators and users validation.

WP 16 Exploitation and technology transfer

Leader

Constellation, United Kingdom

Objectives

This work package focuses on exploitation, technology transfer and standardisation.

Tasks

T16.1: Exploitation. This task will coordinate the technology transfer and exploitation plans and activities. An initial plan is described in detail in the Impact section. The work of WP5.5 will be undertaken in close collaboration with WP14 Dissemination and Communication to reach out to potential users and adopters of the technology, with a clear focus on fostering the uptake and the project results and building a community of business users. The task will define exploitable results, assess the market potential, identify specific technology transfer opportunities and support the implementation of relevant exploitation paths. When possible open source strategies will be followed. Each partner will indicate which exploitable results he will use and all partners will be encouraged to assist in finding end users for the developed software.


T16.2: Standardisation. Activities in T16.2 will concentrate on the standardisation activities, in facilitate the uptake and interoperability of the technology. Focus will essentially be on the definition of new OGF standards, keeping in mind potential interactions with other standardisation and specification bodies like the ones collaborating via http://www.cloud-standards.org/.

T16.3: Cloud Business Models. The goal of this task is to investigate business issues in Clouds, and in particular in CONTRAIL scenarios, suggesting and recommending fundamental design, structure, and mechanisms for SLA management.