PRODUCT HOW-TO: Improve the efficiency of embedded multi-core hypervisor designs

This article will explain some of the history, future hardware trends and emerging usage scenarios of hypervisor technology, then discuss how multicore architectures improve the usability and efficiency of  hypervisors.

Hypervisor history
The concept of a hypervisor that can host multiple 'guest' operating systems on a single hardware platform was pioneered by IBM in its mainframes more than 30 years ago. IBM used hypervisors to enable legacy versions of its operating systems to continue to execute on new hardware platforms.

In addition, IBM software developers used hypervisors as a convenient platform for developing and testing new operating system features. The following is a list of historical uses of hypervisor technology:

* Debugging and testing of privileged code/new architectures
* Debugging and testing of software prior to hardware
* Running distinct and legacy operating systems on the same hardware
* Sandboxes for untrusted software
* Consolidation: improved utilisation of a single computer
* Aggregation: improved utilisation of many computers
* Portability: vary hardware, but keep same OS and applications
* Server provisioning: migration, backup, recovery

Despite its obvious utility, hypervisor technology was largely absent from mainstream computing until the turn of the millennium when VMware demonstrated that virtual machines could run with reasonable performance on commodity PC hardware.

Since then, a variety of hypervisor technologies have come to market, including Microsoft Virtual PC, open source XEN (whose commercial spin-off, XENSource, was recently acquired by Citrix), Linux KVM, Parallels, and Green Hills' Padded Cell. Hypervisor technology was made practical on commodity hardware by a combination of intelligent software and, more recently, hardware acceleration added into the leading mainstream computer architectures.

In 2005, Intel launched its Virtualisation Technology (VT) which dramatically improved hypervisor execution speed on Intel Architecture based platforms. Intel has continued to add additional performance and security features to aid hypervisors. AMD has followed a similar path.

In the PC world, hypervisors are able to implement 'full virtualisation' in which the guest operating systems and their applications are able to run unmodifi ed on the virtual machine. In the embedded world, a number of paravirtualisation (modified guest) solutions have been attempted; however, the performance of embedded microprocessors has precluded full virtualisation.

That may be changing.

Recently, Power.org announced the addition of virtualisation acceleration features into the Power Architecture embedded profi le that may make full virtualisation practical as implementations come to market in the next couple of years.

It is probably a good bet that ARM, MIPS, and other popular embedded processor cores will eventually follow suit. In addition, Intel is promulgating its virtualisation technology to its embedded-class chipset implementations.

More hardware trends
Intel's VT-d (or Directed I/O) technology recently came to market in desktop chipsets, providing an important addition to the virtualisation acceleration story. AMD and Power Architecture use the term 'IOMMU' for the same general capability.

Traditionally, hypervisors needed to emulate I/O devices because each guest operating system could not be trusted to have direct access to DMA and other physical resources which could inadvertently or maliciously affect other guests.

The hypervisor would intercept and emulate all guest I/O requests, creating a performance bottleneck. With IOMMU, the guest can be provided direct access to the I/O device (including DMA); the virtualisation hardware will prevent any I/O access (even from the peripheral itself) from accessing memory outside of the virtual machine. Another major improvement coming to virtualisation hardware is what is referred to as 'extended page tables'.

Traditionally, the hypervisor must intercept every guest attempt to modify system page tables. The hypervisor maintains a 'shadow page table' for each guest, and each guest modifi cation is emulated, accomplishing the desired physical memory partitioning.

In the future, hardware page tables will be extended, adding an additional level so that guest physical memory references will be mapped to a true physical location programmed by the hypervisor.

Thus, guest page table references will no longer need to be emulated, removing what is arguably the largest remaining performance bottleneck in virtualisation today.