Choosing Java hardware requires planning, tough choices

Choosing Java hardware requires planning, tough choices
With the availability of compelling net-centric services and Java hardware, device manufacturers are faced with two questions: At what point do the revenue opportunities enabled by hardware-based Java implementations necessitate a move from a software virtual machine (VM) to a hardware solution? And, what are the implications of migrating from a software VM to a hardware assisted VM?

To deal with these concerns, platform managers need to set the following goals for a successful migration to hardware Java implementations: high performance; design for testability and outsource testable modules; and the preservation of well-known, good systems.

To see how the optimum design is achieved in this context, let's look at two typical implementation approaches. In the first scenario, companion processor suppliers can offer a solution that includes both integrated, proven hardware and software — covering silicon to standard Java APIs — so that developers can create compelling content for their users right out of the box. The supplied product is testable against the profile of choice prior to integration. Furthermore, interfaces between the new components and existing ones are narrow and standard, resulting in local problems and a robust integration.

The problem is usually introduced during the porting of the Java accelerator software stubs, which fail to flush the data when the accelerator traps. Java implementations nearly always use quick versions of standard instructions where the virtual machine performs byte code replacement in the instruction stream.

This can easily cause cache coherency problems. In this instance, the instruction stream then gets corrupted as a fraction of the modified instructions is still in the data cache when execution resumes. What makes the problem difficult is that it is only observable on the customer platform and premises. The faulty code executes in the accelerator mode, traps out and then executes in privileged mode causing a crash in third party VM code.

The first and last defense in cases like this is code inspection. If this fault is not caught during code inspection, it is at least a two week schedule hit. This is because troubleshooting requires an intimate knowledge of the platform hardware, software, and debug tools, and VM and associated tools, as well as the collocation of people and equipment.

In contrast, accelerator suppliers rely on the support and coordination of up to five other internal or external suppliers to complete an integration. For example, the CPU vendor needs to connect to the accelerator, the RTOS team needs to isolate it from other software, the VM team uses it, the application software needs to survive the process, and the Java profile supplier measures the overall impact as a developer would see it.

The involvement of so many parties in the bring-up of an accelerator solution leads to a brittle integration that requires massive cross-organization coordination, and problems that straddle across suppliers, time zones, and geographies.

In the second approach, the performance gains provided by a companion processor are dramatic, since the hardware and software can share knowledge of the basic data structures and the supplier can monitor performance at the system level, as the developers see it. The supplier can then remove bottlenecks, whether they lie in the hardware layer, software layer, or the interface between the two.

In contrast, the performance gain in the accelerator model is killed by the lack of end-to-end ownership of any one supplier. Every provider is constrained to abstractions and interface bottlenecks which builds inertia as vendors face configuration management challenges.

When it comes to Java hardware, vendors sometimes bend and trivialize issues into a form that matches a solution they can readily provide. For instance, it is common practice to ignore the needs of the developer base and application users, and instead address the issue of "accelerating a handful of arithmetic and logic operations in a Java Processing Unit."

A governing principle for a successful integration is to use the right tool for the task at hand. A frequent quote from accelerator vendors is that "a companion processor is harder to integrate and incurs a heavy performance loss due to longer switch times to the host processor." The reality is that a good companion processor system simply never yields to the host to execute Java instructions. In addition, the product is delivered as a fully tested package, where the preferred vendor will also supply all the software required to comply with the profile of choice, whether DoJa, MIDP, or custom extensions as required.

It is much easier to integrate two modules that are independently testable (host and companion processor) than two that are not (host and accelerator). In the companion processor case, bugs can easily be localized to the host, the companion processor VM, or the interface between the two. In the accelerator case, the system is brought up on the target without prior knowledge of its integrity.

In the usual accelerator integration model, a separate third party provides the software VM. When a bug surfaces, it cannot be localized easily. The accountability for integration bug resolution is now split between the customer, the supplier of the accelerator, and a third party software vendor supplying the VM. The problem is further exacerbated by the accelerator hooks being buried in the lowest layers of the customer firmware. Put another way, accountability from the software VM supplier and the accelerator supplier are proportional to the ability of the customer to localize problems in either camp and dismiss its own system code as a culprit. Given the complexity of the debugging cycle of accelerator-based solutions, result-driven project sponsors know that when the software schedule slips, the product slips.

The best way to integrate a companion processor into a working system involves adding an interruptible peripheral on a system on a chip in much the same way as DSP companion processors are included now. The Java companion processor is addressable on a high-speed bus, using its own caches. Both processors share the system bus. Typically, the companion processor is restricted to a certain address range and to a lower priority on the memory bus to preserve the real-time characteristics of the system bus.

From a sample of Java MIDlets running on a Java companion processor, applications had a cache hit rate north of 90% on both instruction and data caches, which limited the impact on the system bus. For further proof that the companion processor does not impact the host, the host can be a greedy bus master with absolute priority on memory accesses.

The companion processor software integration model takes advantage of the common interface between software virtual machines and their native layers. A Java companion processor comes with a software module that attaches to the host and handles the communication between the host and companion processors. In its simplest form, this module redirects file and network operations that were previously handled by the host. To launch applications, the host can either supply a description of the desired application to the companion Java processor, if there is an application browser on the host, or can launch an application manager on the companion Java CPU.

As such, neither the Java applications nor the native layer really know or care whether the virtual machine is software on the host or hardware on the companion CPU since the interface is identical. Customers who want to migrate user-interface code to Java code can do it as a second phase, or in another product based on the same platform.

Accelerator solutions do have one feature worth mentioning: it is easy to decide midstream that the performance gains of the accelerator are not worth the integration trouble. While the integration risk for accelerator solutions is very high, there is always an opportunity to return to the software VM solution. There is only a risk if the platform manager persists in using the accelerator. This is not a derisive statement; some projects have suffered from irreversible decisions as business priorities change.