I/O Virtualization (IOV) & its uses in the network infrastructure: Part 1

At 10G and beyond this dictates the use of an IO virtualization co-processor (IOV-P). By classifying network traffic into flows, applying security rules, and pinning flows to a specific VM (virtual machine) on a specific core on the host, and/or by load balancing various flows into various VMs, the IOV-P enables the overall system to achieve full network performance.

As servers and network appliances in the data centers, and control plane functions in infrastructure equipments, are built around commodity multi-core CPUs " specifically x86 architectures - IO communications are becoming dependent on the system interconnect, such as PCIe. An 8 lane PCIe v2 interconnect can easily support over 10G of network IO traffic.

The increasing use of virtualization in servers, appliances and network equipment means that the underlying IO subsystems explicitly have to support virtualization. Virtualized data center servers and appliances using IOV-P-based intelligent network cards provide each Virtual Machine (VM) with its own virtual NIC, allowing a number of VMs to share a single 10GbE physical NIC (Network Interface Card).

Each virtual NIC can have its own IP and MAC address, and can be assigned to a separate VLAN. To the outside world and to the host sub-system the virtual NIC, appears as a distinct and dedicated NIC. In the same way that multiple VMs running on a multi-core server replaces multiple-physical servers the IOV-P can replace multiple NICs and help replace or simplify the top-of-the-rack switch and the server load balancer.

The result is higher overall performance, lower cost and easier system management using fewer NICs, cables, and switch ports while achieving full network IO performance. Similar benefits apply to network infrastructure equipment when IOV-P is used for intelligent service blades and trunk cards serving the various line cards.

This three series of articles first discusses this new class of network IO virtualization architectures, and in Part 2, its role as a key ingredient in virtualized systems, and finally in Part 3, describes Netronome's new network processor architecture and how it can be used to implement network IO virtualization in a way that provides a combination of high-performance, security and low-power utilization.

Effective Resource Utilization needs Virtualization
As companies grow, their IT infrastructure also grows leading to an increase in the number of stand-alone servers, storage devices and applications. Unmanaged, this growth can lead to enormous inefficiency, higher expense, availability issues, and systems management headaches negatively impacting the company's core business. Smaller servers may have utilization rates of 20% or less.

To address these challenges, organizations are implementing a variety of virtualization solutions for servers, storage, applications, and clients environments. These virtualization solutions can deliver real business value through practical benefits, such as decreased IT costs and business risks; increased efficiency, utilization and flexibility; streamlined management; and enhanced business resilience, and agility.

Enter Server Virtualization
In virtualized severs running VMware, or Xen, the Physical NIC becomes isolated from the guest OS used by application software. The guest OS, such as Windows or Linux, uses a NIC driver to talk to a virtual NIC. The virtualization software (Hypervisor) emulates a NIC for each guest OS. One physical server could have 8 or 16 Virtual Machines, each of which runs a guest OS talking to a virtual NIC.

In addition to allowing multiple guest OS's to share a single physical NIC, the Hypervisor typically emulates an Ethernet (L2) switch connecting virtual machines to physical NIC ports. Implementing virtual NIC functions and virtual switching functions within the virtualization software is performance intensive and adds significant overhead in the networking path. This can reduce 10GbE throughput to 1GbE levels.

Introducing Network IO virtualization
The PCI-SIG IO Virtualization (IOV) working group is developing extensions to PCIe. The first IOV specification maintains a Single PCIe Root complex (SR-IOV) enabling one physical PCIe device to be divided into multiple Virtual Functions. Each virtual function can then be used by a virtual machine, allowing one physical device to be shared by many virtual machines and their guest OSs.