draft-khasnabish-vmmi-problems-00:
Requirements for Mobility and Interconnection of Virtual Machine and Virtual Network Elements
Author(s): Bin Liu, Bhumip Khasnabish
In this draft, we discuss the issues and requirements related to migration, mobility, and interconnection of Virtual Machines (VMs)and Virtual Network Elements (VNEs). We also describe the limitations of various types of virtual local area networking (VLAN) and...
Network Working Group Bhumip Khasnabish
Internet-Draft ZTE USA, Inc.
Intended status: Informational Bin Liu
Expires: June 30, 2012 ZTE Corporation
December 28, 2011
Requirements for Mobility and Interconnection of Virtual Machine and
Virtual Network Elements
draft-khasnabish-vmmi-problems-00.txt
Abstract
In this draft, we discuss the issues and requirements related to
migration, mobility, and interconnection of Virtual Machines (VMs)and
Virtual Network Elements (VNEs). We also describe the limitations of
various types of virtual local area networking (VLAN) and virtual
private networking (VPN) techniques that are traditionally expected
to support such migration, mobility, and interconnection.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 30, 2012.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 1]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Conventions used in this document . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. General Requirements . . . . . . . . . . . . . . . . . . . . . 5
3.1. Data Center Maintenance . . . . . . . . . . . . . . . . . 6
3.2. Disaster Recovery . . . . . . . . . . . . . . . . . . . . 6
3.3. Data center migration or integration . . . . . . . . . . . 6
3.4. Expansion . . . . . . . . . . . . . . . . . . . . . . . . 6
3.5. Load Balancing . . . . . . . . . . . . . . . . . . . . . . 6
3.6. The safety problems in the environment of virtual
machine . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.7. The efficiency of data migration and migration fault
risk processing . . . . . . . . . . . . . . . . . . . . . 7
3.8. How to migrate in the IPV4 and IPV6 mixed environment . . 7
3.8.1. The real-time perception of global network and
storage resources . . . . . . . . . . . . . . . . . . 7
3.8.2. The real-time perception of global network
resources . . . . . . . . . . . . . . . . . . . . . . 7
3.8.3. The real-time perception of global storage
resources . . . . . . . . . . . . . . . . . . . . . . 8
3.9. General requirements of migration state . . . . . . . . . 8
3.9.1. Migration schedule foundation . . . . . . . . . . . . 8
3.9.2. Migration authentication . . . . . . . . . . . . . . . 8
3.9.3. Migration ability consultation . . . . . . . . . . . . 8
3.9.4. Standardization of migration state . . . . . . . . . . 8
3.10. The selection of migration . . . . . . . . . . . . . . . . 10
3.10.1. Two requirements with different network
environment and protocol . . . . . . . . . . . . . . . 10
3.10.2. Requirements for a wider range of virtual machine
live migration . . . . . . . . . . . . . . . . . . . . 10
3.11. The part of traffic roundabout analysis on the VM WAN
migration . . . . . . . . . . . . . . . . . . . . . . . . 11
3.11.1. VM migration requirements . . . . . . . . . . . . . . 11
3.11.2. Scene analysis . . . . . . . . . . . . . . . . . . . . 11
3.12. Robustness . . . . . . . . . . . . . . . . . . . . . . . . 11
3.12.1. Robustness of VM . . . . . . . . . . . . . . . . . . . 11
3.12.2. Robustness of VNE . . . . . . . . . . . . . . . . . . 12
3.13. Requirement on DCI interconnection fabric . . . . . . . . 12
3.14. Requirement of Cloud service Virtualization . . . . . . . 13
3.14.1. Requirement of logical element . . . . . . . . . . . . 13
3.14.2. Requirement of Resource Allocation Gateway function . 13
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 2]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
3.14.3. Requirement of specifications and performance . . . . 14
3.14.4. Requirement of fault tolerance capability . . . . . . 14
3.14.5. Network model . . . . . . . . . . . . . . . . . . . . 15
3.14.6. Types& Applications of VPNs interconnection
between DCs which provide cloud services . . . . . . . 15
3.14.6.1. Types of VPNs . . . . . . . . . . . . . . . . . . 16
3.14.6.2. Applications of L2VPN in DC . . . . . . . . . . . 16
3.14.6.3. Applications of L3VPN in DC . . . . . . . . . . . 17
3.14.7. VN Requirement . . . . . . . . . . . . . . . . . . . . 17
3.14.8. Requirement of Mobility . . . . . . . . . . . . . . . 17
3.14.9. Mobility Requirement . . . . . . . . . . . . . . . . . 18
3.14.9.1. Summarization of Mobility . . . . . . . . . . . . 18
3.14.9.2. Problem Statement . . . . . . . . . . . . . . . . 18
3.15. MAC, IP, ARP Explosion . . . . . . . . . . . . . . . . . . 19
3.16. Suppressing the flooding within VLAN . . . . . . . . . . . 19
3.17. Convergence and multipath support . . . . . . . . . . . . 19
3.18. Multicast processing . . . . . . . . . . . . . . . . . . . 19
3.19. Requirement of others . . . . . . . . . . . . . . . . . . 20
4. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5. Security Considerations . . . . . . . . . . . . . . . . . . . 21
6. IANA Consideration . . . . . . . . . . . . . . . . . . . . . . 21
7. Normative References . . . . . . . . . . . . . . . . . . . . . 21
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 3]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
1. Introduction
In this document, the requirements to migrate the virtual machines
are summarized, respectively, from the necessary conditions for
migration, business needs, state classification, security,
efficiency, and the migration program are described. Then list the
requirement for VM migration in the current IPV4 and IPV6 mixed
environment. On the choice of the migration solution, the
requirements of techniques either on large-scale Layer-2 network or
on the segmented IP network are discussed. The former is often used
for frequent data migration with high requirements on data security,
such as data migration and backup in the bank, etc.; the latter is
used for data migration for personal or mobile users, or between
different cloud service providers. Because there are more and more
pressing needs in the future to build the distributed PaaS across
different cloud service providers, a unified management platform will
offer very high efficiency for users as well as cloud service
providers. Secondly, summarize the requirement of virtual networks,
such as VM migration, visual network, DCI modes, etc; Finally, simple
statements are made for common ways of interconnection in IDC.
1.1. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
2. Terminology
o DCB: Data Center Border Routers
o DC GW: Data Center Gateway
o DCS: Data Center Switch
o IP VPN: Layer 3 VPN, defined in L3VPN working group
o OTV: Overlay Transport Virtualization
o RA GW: Resource Allocation Gateway
o ToR: Top of the Rack
o VLAN: Virtual Local Area Networking
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 4]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
o VM: Virtual Machine
o VMMI: virtual machine mobility and interconnection
o VN: Virtual Network
o VNE: Virtual network element. For example, a virtual switch on a
server virtualization platform works as a virtual network element
o VPN: Virtual Private Network
o VPLS: Virtual Private LAN Service
3. General Requirements
Data center architecture consists of the following components:
Gateway (Data Center Gateway, Resource Allocation Gateway), Core
Router / Switch, (Aggregation layer switch), Access layer ToR switch,
Visual switch, and interconnecting network between DCs, Servers,
firewall system, etc.
Why VM and VNE need to be migrated?
VMS through migration management can save users money, maintenance
costs and upgrade costs. The volume of previous server is relatively
large; and the volume of the present server is relatively small. The
migration technology allows users to simultaneously use a single
server to replace various numbers of the previous servers, thus saves
the user a lot of room space. In addition, the server of virtual
machine has a unified "virtual hardware", unlike the previous server
which has a number of different hardware resources. After migration,
the server can be managed in a unified interface. Though using some
of the virtual machine software such as high availability tools
provided by VMware, when the server shutdown due to various failures,
it can automatically switch to another virtual server in the network
for non-disruptive operation. In short, migration has the advantage
of lowering operations costs, simplifying maintenance, improving
system load balancing, enhancing system error tolerance and
optimizing system power management.
Overall, VM migration faced with the following questions:
How to accommodate a large number of tenants in each isolated network
in the data center;
From one DC to another within one Admin domain, how to ensure the
necessary conditions in migration, how to ensure a successful
migration without disruption to services; how to ensure the success
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 5]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
of rollback when problems occur;
From one domain to another domain, how to solve the problem of
communication between domains;
In L2 network, VXLAN is used to resolve VLAN expanding problem&#
65294;IP based technology is used to resolve migration In L3 network
and VPN technology is used to carry L2 and L3 traffic across the IP/
MPLS core network.
3.1. Data Center Maintenance
The servers or applications in the data center architecture should
maintain uninterrupted service in the migration.
3.2. Disaster Recovery
In the face of some major natural disasters, business-critical data
center applications can be migrated to other data centers in advance.
3.3. Data center migration or integration
As a data center migration or integration needs, some of the
applications can be migrated without interruption from a data center
to another data center.
3.4. Expansion
As for the considerations of address resources, cooling and physical
space in the primary data center, some of the virtual machines can be
migrated to the backup data center.
3.5. Load Balancing
In the migration of virtual machines between data centers, users are
provided with the nearest calculation principle of "follow the sun",
or multi-site load balancing requirement. In addition, for reducing
energy, cooling costs and other considerations, the virtual machines
can be integrated into less dynamic data center, which is the future
trend of "green" data center.
3.6. The safety problems in the environment of virtual machine
Some problems, such as the operating environment of virtual machine,
how to prevent the invasion and how to detect in normal operation or
VM migration, are not concerned at present.
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 6]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
3.7. The efficiency of data migration and migration fault risk
processing
A lot of invalid data does not need to migrate. Consider to
streamline before the migration. Such as:
Incremental migration:
Only migrate the differentiated data between two parties.
Fault risk processing:
If migration between different, heterogeneous database system, such
as transfer data from ORACLE database in linux system to SQL Server
database in Windows system, it is necessary to define the security
and policy when fault happen: the processing would be very slowly
when database migration operation fail and need to roll back all the
database processing. It is necessary to define the database roll
back policy ,such as maximal waiting time between the different type
database.
3.8. How to migrate in the IPV4 and IPV6 mixed environment
A lot of issues need to be considered for migration in this mixed
environment, since IPV4 and IPV6 will coexist for a long time.
3.8.1. The real-time perception of global network and storage resources
In the current system, there exists such contradiction: network
resources and storage resources do not match, that is, the
availability of network system resources and the real-time
requirements of virtual machines/ storage system resources in the
data center are often inconsistent. From the global the scale, the
storage resources in the distributed data center system cannot be
used more efficiently. In turn, network resources cannot be used
more efficiently. Therefore, a management model needs to be
established, that can be informed of system-wide network resources
and storage resources, and dispatch them. The management model can
be integrated into the framework of virtual machine migration under
DMTF standardization.
But how do we learn the system-wide network resources and storage
resources? We need means and protocols to solve these problems.
3.8.2. The real-time perception of global network resources
One important reason that prevents us informed of system-wide
availability of network resources is, in the transition process of
IPV4 to IPV6 network, the different types of traditional tunneling
gateway cannot communicate with each other. In the transition
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 7]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
network using tunneling transition technique, the connections between
the subnets and the backbone network are achieved through the
tunneling gateway. The establishment of the tunnel varies with
different gateways. Tunneling communication can be established
between the same types of tunneling gateway. The tunnels between
different types of tunneling gateway cannot be established. Here
paper presents a multi-tunneling VPN gateway solution to resolve the
problem that tunnels cannot be established between heterogeneous
tunneling gateways.
3.8.3. The real-time perception of global storage resources
The access of data center virtual machine / storage system resources
can be done well, as long as the relevant API interface, format, and
communication protocols are defined.
These virtual machine / storage system resources in the global scope
are registered, and reported to the resource management system in the
cloud system.
Eventually, the resource management system in the cloud system is
informed of the whole network resources, which lays the foundation
for the determination of the follow-up ability matching.
3.9. General requirements of migration state
3.9.1. Migration schedule foundation
The manager should be able to confirm which data center currently
need to interconnect and migrate data in the whole network.
3.9.2. Migration authentication
Use authentication on parties of the network related with the
migration, storage resources authentication, firewall authentication.
3.9.3. Migration ability consultation
After passing the authentication, related parties need to determine
whether they have the ability to migrate, including network bandwidth
resources, storage resources, resource pool scheduling and so on.
3.9.4. Standardization of migration state
Related parties should be able to understand the state of each other:
Global detection -> authentication processing -> capability
negotiation->session establishment -> initialization instance ->
establish the beginning stage of life -> begin migration -> migration
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 8]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
& migration exception handling -> finish migration -> the end stage
of life -> destruction of instances - > Global detection
+------------------------+
| \|/
| +------------------+
| | Global detection |
| +------------------+
| |
| \|/
| +------------------+
| | authentication |
| | processing |
| +------------------+
| |
| \|/
| +------------------+
| | capability |
| | negotiation |
| +------------------+
| |
| \|/
| +------------------+
| | session |
| | establishment |
| +------------------+
| |
| \|/
| +------------------+
| | initialization |establish the beginning stage of life
| | instance | |
| +------------------+ |
| \| |
| +---------------| |
| | /| |
| | | |
| | \|/ |
| | +------------------+ |
| | | begin migration | |
| | +------------------+ |
| | | |
| | | |
| | \|/ |
| +------------+ |
| | exception |/ Y migration |
| | processing |--- exception? |
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 9]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
| +------------+\ |
| | |
| |N |
| | |
| \|/ |
| +------------------+ |
| | finish migration | |
| +------------------+ |
| | |
| \|/ |
| +------------------+ \|/
| | destruction | the end stage of life
| | of instances |
| +------------------+
| |
+------------------------+
Figure 1: migration state
3.10. The selection of migration
3.10.1. Two requirements with different network environment and
protocol
Currently the large-scale Layer--2 interconnect technique is mainly
used in the migration of virtual machine, but there are also the
needs for the Layer-3 migration. These two technologies apply for
different implementation scenario. The former is often used for
frequent data migration with high requirements on data security, such
as data migration and backup in the bank, etc.; the latter is used
for data migration between data centers without a VPN network, or
between different service providers. This demand really exists.
Because of user requirements on the establishment of a unified
management platform, it will become more and more important to build
the distributed PaaS across different cloud providers. No user is
willing to maintain too many independent platforms. At the same
time, sharing resources across the data center will become a trend.
As a result, it will become very cumbersome for data center managers
to build a large number of VPN connections for all data centers. So
there will be a portal operator, who take care of all t internal VPN
connections between the clouds and unify the scheduling of data
migration, or use the partitioning program.
3.10.2. Requirements for a wider range of virtual machine live
migration
a) Virtual machine live migration across IPv4 / IPv6 network; b)
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 10]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
Solutions applying to the evolution from IPv4 to IPv6; c) Virtual
machine live migration based on mobile IP. Virtual machines can also
be mobile. Need to achieve transparent live migration of virtual
machines cross-IPv4 / IPv6 network, and be able to apply to a variety
of application environments in the early, mid-and late evolution of
IPv4 to IPv6. Scenarios: This application is suitable for the mobile
applications of small users or home users. The complexity of the
network is full shielded from the user, as long as the source and
destination have an IPV6 address. It can be the application scenario
of virtual machine migration in the Layer-3 network environment. It
is suitable for individuals skeptical about the security of cloud.
3.11. The part of traffic roundabout analysis on the VM WAN migration
3.11.1. VM migration requirements
VM MAC / IP keep the same, migration in the L2 network.
3.11.2. Scene analysis
VM migrates from the IDC in metro A to metro B. There is almost no
traffic roundabout for users within the metro (such as user a). For
access to IDC business by WAN, such as metro C, the client traffic
must first access VM-A gateway after VM migration, and then sent to
the migrated VM through the Layer-2 tunnel.
3.12. Robustness
3.12.1. Robustness of VM
When a VM is running, it will inevitably encounter all kinds of
problems: CPU is overloaded, memory is not enough, disk space is not
enough, the program is not responding, database write fails, the file
system fails, etc. If any of the problems is not resolved, it will
lead to the collapse of the VM. The VM system should be able to
start emergency treatment: taking the snapshot of all data in the VM
and coping to a blank VM in another server, or coping to another
blank VM in current server, in order to achieve the purpose of
services provided by the original VM can continue running without any
disruption. In the process of taking snapshot, the snapshot can be
either stateful or stateless, depending on the status, nature, and
function of the owner which various data belongs to in the VM, and
the strategy of replication. For example, the data in the database
is taken a stateful snapshot, because the database itself has the
ability to record the database running state.
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 11]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
3.12.2. Robustness of VNE
When a VNE is running, it will also inevitably encounter all kinds of
problems: CPU is overloaded, memory is not enough, MAC table is full,
forwarding table is full, the program is not responding, etc. If any
of the problems is not resolved, it will lead to the collapse of the
VNE. The VNE system should be able to start emergency treatment:
taking the snapshot of all data in the VNE and coping to another
blank VNE in current server, in order to achieve the purpose of
services provided by the original VNE can continue running without
any disruption. In the process of taking snapshot, the snapshot can
be either stateful or stateless, depending on the status, nature, and
function of the owner which various data belongs to in the VNE, and
the strategy of replication. For example, the state of the protocol
is taken a stateless snapshot, and forwarding tables of protocol-
independent are taken a stateful snapshot, because the protocol is
running time-dependently, if the time is different, the state should
be different, so the state of the protocol does not have to copy all.
3.13. Requirement on DCI interconnection fabric
This fabric should be open and transparent, which can be achieved by
simple extension on some of the existing technology. The program
should have a strong openness and compatibility, easy to deploy.
The negative influence of ARP, MAC and IP entry explosion on the
individual network which contains a large number of tenants should be
minimized by DC and DC-interconnect technologies.
The link capacity of DC and DC-interconnect should be effectively
utilized.
Traffic should be forwarded on the shortest path between two VMs
within the DC or across DCs.
Efficient traffic forwarding requires effective utilization of the
link capacity of DC and DC-interconnect, and traffic forwarding on
the shortest path between two VMS within the DC or across DCs.
Inter-DC connectivity must be supported because of these factors:
Support east-west traffic between customer' applications located in
different DCs
Management across the DC
Mobility of VM migration across the DCs
Many mature VPN technologies can be used to provide connectivity
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 12]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
between DCs, or the extension of VLAN and virtual domain between DCs.
3.14. Requirement of Cloud service Virtualization
3.14.1. Requirement of logical element
Resource Allocation Gateway
Network service providers provide virtualized basic network resources
for tenants between tenants' sites and tenants' data center. In the
data center, data center provider provide Virtualization computing
and Virtualization storage resources. The gateway's role is to
allocate virtualized resources. These resources are divided into
three categories: network resources, computing resources and storage
resources. The gateway compares network resources with storage
resources, finds out the corresponding relations, and achieves
globally reasonable resource scheduling. DC GW's function is a
subset of RA GW functions.
Data Center Gateway
The gateway provides access to the data center for different users in
the outside world, including Internet and VPN connections. In the
current DC network model, DC GW may be a router with virtual routing
capabilities, or may be a PE of IPVPN/L2VPN. Core Nodes which
perform the DC GW role also play roles as Internet connectivity,
inter-DC connectivity and VPN support.
Core Router / Switch
High-end core nodes of switch with routing capabilities located in
the core layer, connecting aggregation layer switches.
Aggregation layer switches The switch aggregates traffic from the ToR
switches and forwards downstream traffic. The switch can be a normal
aggregation switch, or multiple switches virtualized into a single
stack switch.
Access layer ToR switch
ToRs are usually dual-homed to the parent node switch.
Visual switch
A virtual software switch which runs on blade servers.
3.14.2. Requirement of Resource Allocation Gateway function
Data center and network providers provide virtualized computing,
storage and network resources services. Tenants are identified by
the overlapping addresses, and share a pool of storage and network
resources. Therefore, a virtual platform is needed, including
virtual machines, virtual services, virtual storage and virtual
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 13]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
networks. What tenants see is a subset of the above four. The
virtual platform is built on the framework of the physical network,
physical servers, physical switches and routers, physical storage
devices. Through the virtual platform, users are achieved global
resource scheduling in the whole system.
According to the collected network-wide computing, storage and
network resource information, resource allocation gateway allocates
appropriate computing, storage and network resources for tenants
according to certain rules.
Resource allocation gateway needs redundant backup, in order to
prevent a single point of failure. The global resource information
and scheduling information between resource allocation gateway and
backup resource allocation gateway need real-time backup.
It can provide automatic match and scheduling of the virtualized
resources, which are dynamic and adjust according to operating
conditions. It can optimize utilization of the computing resources,
network resources (including IDC interconnection resources and IDC
internal routing & switching resources) and storage resources. It
should consider the optimization of the network path routing in
network matching. Routing selection is based on: the degree of
matching between the required bandwidth and the bandwidth which can
be provided, the shortest path, service level, user level. These
factors need to be set priorities in decision-making.
3.14.3. Requirement of specifications and performance
It should be able to support a large number of tenants sharing data
center resources, and need to support VLANs much larger than 4K. For
example, there are a number of VPN applications (VPLS or IP VPN)
which serve more than 10K customers, each requiring multiple VLANs,
then 4K VLANs is not far enough to use.
It should be able to guarantee high quality of service, and ensure a
large number of network connections are not interrupted. The
connectivity should meet carrier-level requirements.
3.14.4. Requirement of fault tolerance capability
In the event of an error, it is able to quickly recover from an error
condition. Error recovery includes network fault recovery, computing
power recovery, VM migration recovery and storage recovery. Among
them, the network fault recovery capability and computing power
recovery are the foundation of VM migration recovery and storage
recovery.
Network fault recovery: Once the virtual network connectivity sends
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 14]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
an error, it can automatically trigger alarms and handling, and can
rapidly enable a backup virtual network to recovery;
Computing ability recovery: Once the computing fails, the detection
mechanism is needed to find the problem and services can be scheduled
to backup virtual machines and virtual services;
VM migration recovery: In the event of migration failure, it is
needed to automatically restore to the original state of the virtual
machine and ensure that does not affect the users;
Storage recovery: In the event of storage failures, it is needed to
automatically find a backup virtual storage resource and enables it
immediately.
The above fault tolerances should be able to have enough short
response time and recovery speed, and minimize service delays and
malfunctions.
After the VM migration, it is needed to consider the impact on the
switching network, such as whether the new network environment will
have the problem of insufficient bandwidth. Although at the
consultation phase before the migration, there will be an initial
judgment, but it cannot guarantee no problem at all after the
migration; In addition, if the destination Data center needs to
activate the standby server and network resources, we should consider
extend the server resources and network resources.
In some cases, some routing policies need to be done on the network
and servers after migration.
3.14.5. Network model
DC has its own private network for the interconnection between the
data centers, or uses other network service provider's network
resources at the sametime.
DC service and WAN network services all are achieved by the same
company or organization;
DC service and WAN network services are achieved by two different
companies or organizations;
3.14.6. Types& Applications of VPNs interconnection between DCs which
provide cloud services
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 15]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
3.14.6.1. Types of VPNs
Layer3 VPN
BGP / MPLS IP Virtual Private Networks (VPNs) (BGP / MPLS IP Virtual
Private Networks (VPNs))
RFC 4364
Layer2 VPN
PBB + L2VPN
TRILL + L2VPN
VLAN + L2VPN
NVGRE (draft_sridharan_virtualization_nvgre)
PBB VPLS
E-VPN
PBB-EVPN
VPLS
VPWS
3.14.6.2. Applications of L2VPN in DC
L2 interconnecting across physical regions has become a standard way
of DC interconnection. VPN technology is used to carry L2 and L3
traffic across the IP/MPLS core network. This technology can also
used in the same DC to provide for scale expansion or interconnection
across L3 domains. VPLS has acted as an important role in IP / MPLS
WAN and supplied transparent LAN services. IP VPN, including BGP /
MPLS IP VPN and IPSec VPN, has been used in a common IP / MPLS core
network to provide virtual IP routing instances.
The implementation of PBB + L2VPN can take advantage of some of the
existing technology. It is flexible for VPN network in cloud
computing and can provide a sufficient number of VPN network
resources, which is much larger than the 4K VLAN mode of L2VPN and
the effect is similar with VXLAN. The use of PBB makes us not only
have access to more than 16M virtual LAN instances, but also can
separate the customer and provider domains by isolated MAC address
spaces.
Using PBB encapsulation, have a major advantage, VM's MAC address
will not be processed by ToRs and Core SWs, so Mac table size of ToRs
and Core SWs may be reduced two orders of magnitude, the specific
number is related with the number of virtual machines in each server
and VM virtual interfaces.
One solution to solve problems in DC is to deploy other technologies
in the existing DC network. A service provider can separate its
domains of VLAN into different VLAN islands, in this way each island
can support up to 4K VLANs. Domains of VLAN can be interconnected
via VPLS, at the same time, use DC GW as a VPLS PE.
If retaining the existing VLAN-based solutions only in VSw, while the
number of tenants in some VLAN islands is more than 4K, the service
provider need to deploy VPLS deeper in the DC network, that is, start
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 16]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
to provide support for L2VPN from the ToRs, and use the existing VPLS
solutions to enable MPLS for the ToR and DC core elements.
3.14.6.3. Applications of L3VPN in DC
IP VPN technology can also be used for data center virtualization, if
each tenant of requiring L3 virtualization is assigned a different IP
VPN instance, we can achieve in the DC network to provide multi-
tenant L3 virtualization support.
There are many advantages when using IP VPN as a L3 virtualization
solution within DC compared to existing virtual routing DC
technology, for example:
Supports many VRF-to-VRF tunneling options containing different
operational models: BGP/MPLS IP VPN, IP or L3 VPN GRE, etc.
The connection of IP VPN instances used in Cloud services below the
WAN can be IP VPN directly involved in the WAN, and so on.
3.14.7. VN Requirement
VN is composed by the virtual IDC network, virtual DC internal
switching network. These virtual networks are built on the basis of
the physical networks. VM migration is not affected by the physical
network. As long as it is within the scope of the VN, it is free to
migrate with the necessary conditions; In addition, network
architecture and forward switching capacity should match between the
source network and destination network, without concern for physical
network. The physical characteristics of the network, such as VLAN,
IP subnet, L2 protocol, QoS and so have been abstractly called the
logic concept of VN; Because VM operating environment is VN, like VM
has the associated logic concepts, such as the CPU process, I / O,
memory, Disk, etc., VN also has a corresponding set of logical
concepts.
VNs are isolated from each other. Its internal VMs communicate with
their own internal address, send and receive Ethernet packets.
VN do not have ties to their specific implementation, the
implementation can be Internet, L2VPN, L3VPN, GRE, etc. From the VN
layer, IP can be used to make that distinction. A firewall is needed
to go into the VN, ACL and other security policies are needed on the
access layer.
3.14.8. Requirement of Mobility
It is needed to consider the support for mobility, such as a VM can
be easily migrated repeatedly between a number of DC (greater than
2). It is needed to consider free migration program in the IPV4 and
IPV6 VPN environments. This part of the demand should be considered
in the processing business of DC GW.
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 17]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
VMs in the resource pool should have a good mobility. They can be
moved between different servers within the same DC or from the DC to
a remote DC. This has many advantages, for example: In the event of
a natural disaster, the VM is migrated to a safe location of DC in
order to reduce downtime. The VM can also be migrated to sites to
reduce costs, where the rent or the electricity fee is cheap. When
the VM is migrated to a new location, it should maintain the existing
client sessions. VM's MAC and IP address should be preserved and the
state of the VM sessions should be copied to the new location.
Some virtual machine migration tools now widely used require that
management program on the source server and destination server is
directly L2 connected. Its purpose is to facilitate the
implementation of VM migration, such as VMware's VMotion virtual
machine migration tool.
Firstly, A VMotion ELAN may need to provide protection and load-
balancing across multiple DC network. Secondly, in the current
VMotion procedure, the new location of the VM must be part of the
tenant ELAN domain. When a new VM is activated, a Gratuitous ARP is
sent, and the MAC FIB entries in the "tenant ELAN" are updated to
direct the traffic for that VM to the new VM location. Thirdly, if
the path needs IP forwarding, the accessibility information of VM
must be updated to the shortest path information to the VM.
3.14.9. Mobility Requirement
3.14.9.1. Summarization of Mobility
Mobility means to move a VM from one server to another server within
the DC or in a different DC, while maintaining the original VM's IP
and MAC address unchanged. VM mobility does not change the VLAN/
subnet connected VM, and requires VLAN be extended to the new
location of VM.
In summary, the seamless mobility solution in DC is based on IP
routing, BGP / MPLS MAC-VPN, BGP / MPLS IP VPNs and NHRP.
3.14.9.2. Problem Statement
The following statement discusses the problems faced in seamless VM
mobility.
The first problem is that the source server and destination server in
VM migration may be located in different data centers, which needs to
extend the layer-2 network. There are islands formed by the same
VLAN in different data centers.
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 18]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
The second problem is that the optimal forwarding in a VLAN for VM
mobility may involve multiple data centers.
The third problem is what the optimal intra-VLAN forwarding mode is.
The forth problem is what the optimal routing mode is.
3.15. MAC, IP, ARP Explosion
Each blade server supports 16-40 VMs, or even more, and each one has
its own MAC and IP addresses. Network devices in the data center
will encounter many problems for their conventional framework and
communication environment because of accommodating such a huge number
of IP, MAC addresses and ARP. Disk, memory, FDB table, MAC table,
convergence time will increase. In order to accommodate the mass of
the servers, it may affect the network topology, for example, is the
Fat tree topology or a conventional network topology of the network
device.
The amount of ARP packets will grows with not only the number of
virtual L2 domains or ELANs which is instantiated on server but also
the number of VMs in that domain. There will be questions such as:
overload of ARP entries on Server/Hypervisor, exhaustion of ARP
entries on the Routers/PEs and processing overload of any other L3
service appliances.
These problems will lead to flood explosion throughout layer 2
switching network.
3.16. Suppressing the flooding within VLAN
From the business perspective, DC operators should try to reduce the
flooding of broadcast, multicast and unknown unicast frames within
VLAN caused by the improper configuration.
3.17. Convergence and multipath support
STP is used to solve the broadcast storm problem in the loop, but it
also brings the problem of inefficient use of resources and network
oscillation. The solutions include switch virtualization, or the
imperfect TRILL and SPB.
3.18. Multicast processing
STP bridge is often used to perform IGMP and/or PIM snooping to
optimize multicast data delivery. However, this snooping is
performed by local STP topology. All traffic goes through the root
bridge for each bridge. This may lead to sub-optimal multicast
traffic transmission. In addition, each customer multicast group is
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 19]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
associated with the forwarding tree network throughout the Ethernet
switching network. Efficient Layer 2 multicast must be provided in
the solution.
3.19. Requirement of others
Packet encapsulation
It is needed a way similar to overlay address to implement VN.
Overlay address can be reflected by VXLAN or the I-SID of PBB +
L2VPN. The overlay address as an identifier corresponds to every VN
instance. The implementation model requires edge switch or router as
DC GW for the encapsulation and decapsulation of tunnel packets. The
various VNs within the DC rely on overlay address to distinguish and
separated from each other. Each VN also contain 4k VLAN for its
internal use. The data go into DC interconnection through DC GW and
are encapsulated through strip for transmission
Routing control
The processing mechanism of different types of packets such as
unicast, multicast and broadcast, the processing mechanism of ARP
packet, load-sharing mechanisms need to be further clarified.
Security and authentication
In the migration process, it is needed to consider security issues,
such as how to solve traffic roundabout issue, how to ensure that the
firewall functionality during the migration process will not be
weakened
Connectivity
Consider whether to join VRF real-time in unit of HOST, or to join
the VRF in unit of VM, and what elements are needed to constitute
this connectivity, such as bandwidth, quality of service and so on.
Computing resources
Consider what kind of format to reflect the statistical units model
of computing resources in the VM, VN environment.
Storage resources
Consider what kind of format to reflect the statistical units model
of storage resources in the VM, VN environment.
4. References
[PBB-VPLS] Balus, F. et al. "Extensions to VPLS PE model for
Provider
Backbone Bridging", draft-ietf-l2vpn-pbb-vpls-pe-model-
04.txt (work in progress), October 2011.
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 20]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
[VM-Mobility] Raggarwa, R. et al. "Data Center Mobility based on
BGP/MPLS, IP Routing and NHRP", draft-raggarwa-data-center-
mobility-01.txt (work in progress), September 2011.
[VPN Applicability] Nabil Bitar. "Cloud Networking: Framework and
VPN Applicability",
draft-bitar-datacenter-vpn-applicability-01.txt, October 2011
[VXLAN] M.Mahalingam. "VXLAN: A Framework for Overlaying Virtualized
Layer 2 Networks over Layer 3 Networks",
draft-mahalingam-dutt-dcops-vxlan-00.txt, August 26, 2011
[NVGRE] M. Sridharan. "NVGRE: Network Virtualization using Generic
Routing Encapsulation",
draft-sridharan-virtualization-nvgre-00.txt, September 2011
[NVO3] Thomas Narten. " NVO3: Network Virtualization", l2vpn-9.pdf,
November 2011
5. Security Considerations
--[Note] Will be added in future.
6. IANA Consideration
The extensions of this draft is baed on DC environment.
7. Normative References
[RFC 4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, February 2006.
Authors' Addresses
Bhumip Khasnabish
ZTE USA, Inc.
55 Madison Avenue, Suite 160 Morristown, NJ 07960
USA
Phone: +001-781-752-8003
Email: vumip1@gmail.com
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 21]
Internet-Draft Mobility and Interconnection of VM & VNE December 2011
Bin Liu
ZTE Corporation
15F, ZTE Plaza, No.19 East Huayuan Road,Haidian District
Beijing 100191
P.R.China
Email: liu.bin21@zte.com.cn
Bhumip Khasnabish & Bin Liu Expires June 30, 2012 [Page 22]
