Volume rendering adds realistic layers to 3-D graphics

Volume rendering adds realistic layers to 3-D graphics
TOKYO — High-end workstation manufacturers and silicon vendors are gearing up to give software developers the processing power to take 3-D graphics to the next level of realism. This year, Mitsubishi Electric will announce what it calls a ground-breaking device that will allow software developers to cut through 3-D objects and view interior structures as if using a virtual knife — all in real-time. Similarly, Silicon Graphics Inc. (SGI) is working to retrofit its entire workstation line this year with 3-D texture-mapping capabilities to provide the muscle for its new "volume rendering" application programming interface.

Mitsubishi and SGI have similar goals: to capture real objects using traditional scientific methods like ultrasound, render a 3-D image with "mass," then uncover hidden interiors. They disagree, however, over whether it will take a giant leap in specialized graphics processing to handle the extraordinary load or whether these systems can exploit evolutionary advances in more-conventional polygon-based rendering devices.

Despite the different hardware approaches, the companies are discussing ways to merge their APIs with the ultimate aim of creating a software interface under the Farenheit project, a joint Microsoft-SGI initiative expected to unify disparate graphics APIs.

Volume rendering has been bandied about for more than a decade in academic and R&D circles. The idea is to capture an image — say a CAT scan of someone's brain — and assign the correlating data into a set of cubes. That breaks from the conventional 3-D rendering approach of placing polygons on flat wire frames that can be twisted, rotated or otherwise contorted but can never truly simulate the interiors of an object. In essence, 3-D rendering itself goes 3-D with volume rendering. Now, a new generation of hardware is reviving the old concept.

"You have architectures that are small enough, memory that is cheap enough and now the APIs," said Steve Artim, marketing manager for Mitsubishi Electric America Inc. (Cambridge, Mass.). "Conceivably this can be part of a mixed-media setting in which low-end PCs will have this capability."

And while most activity is centered in the scientific community, observers said, it will inevitably trickle down into mainstream apps like games. Observers said volume rendering is now being used in games in some instances, such as rendering a rocket blast. It's been publicly discussed by mainstream graphics vendors in the last year as other techniques, like image-based rendering proposed by Microsoft's Talisman project several years ago, have been cast aside.

The pockets of specialized fields that use volume rendering heavily — such as geophysical surveys for oil and gas exploration — rely largely on the CPU to process the load, a painfully slow method. Moreover, software introduced last year for the medical, geophysical and microscopy fields is fueling the need for workstations outfitted with volume rendering.

"The software applications are the driver," said said Jamie Jacobs, general manager for Mitsubishi's volume-graphics unit. "But they either need dramatic workarounds or are just suffering from being so far from real-time. That creates a market opportunity for us."

How to configure the hardware to handle the extraordinary processing load has now taken center stage. Many agree that the memory subsystem must be bolstered, and perhaps organized in a way to minimize access conflicts among different data types. But the kind of processing engine that will be needed is up for debate.

Mitsubishi advocates a DRAM-laden PCI card using a dedicated volume-rendering chip, dubbed the vp500. Three years in the making, the card is to begin sampling in June and will take an NT workstation with the right disk space and enough main memory into the volume-rendering realm.

SGI, meanwhile, argues that volume rendering should mesh with today's polygon-based rendering and that both can be handled by upgrading workstations to include hardware 3-D texture mapping.

Last year, SGI added 3-D texture mapping to two of its high-end workstations — the Onyx 2 and Octane — and plans to add the capability to its entire line by year's end, said Shawn Hopwood, group manager for graphics APIs at Silicon Graphics (Mountain View, Calif.).

"We try to build machines that hit the sweet spot of the general graphics market," he said. "Advanced 3-D texture maps are a new form of generalized graphics starting to appear on workstations. It's a way to have accelerated volume rendering without a dedicated rendering chip, and you can have 3-D texture mapping for other applications."

But Mitsubishi said SGI's approach does not live up to the image quality enabled by using a dedicated processor. "SGI has been advocating trying to handle volume rendering through polygon graphics, and I think it will be hard for them to move from that. They seem oriented to use the Volumizer [API] to map a volume problem through surface-rendering techniques," Jacobs said.

With SGI's Volumizer API, introduced last year, software developers can create "volumetric primitives" analogous to primitives used in polygon rendering, like a cone or a sphere. These primitives create volume images that can be sliced at any angle and represented as 3-D texture maps in real-time.

"Volume rendering isn't a specialized operation anymore," Hopwood said. "We've been able to do one API that merges what you can do with volumes and surfaces in the same scene." With both volume and surface to work with, software developers can create, for instance, a polygon-based scalpel to cut through a portion of volume-rendered brain tissue.

Avoiding collisions
First, light rays are passed through the 3-D data set and samples are composited across each. At this stage, the graphics engine performs the trilinear interpolation to generate missing data between the voxel points that poke through a "base plane," which forms the basis of the viewer's image. A second classification step determines which values to make visible.

For real-time operation, the memory subsystem must be organized to access 256 cubes of data at 1 Gbyte/second. So a Mitsubishi algorithm partitions the memory to avoid access collisions, Jacobs said.

The add-in board needs several banks of 133-MHz SDRAMs, which are just going into volume production. There are 128 Mbytes to capture image data, 16 Mbytes used for storing intermediary results of algorithmic calculations and another 16 Mbytes that hold the output image. The system requires another 128 Mbytes of main memory.

The system itself runs on Windows NT and several hard-disk drives with Gbyte capacity, Jacobs said. "The processor need not be that special," he said; Pentium-class will do.

Mitsubishi has its own API for the vp500, though the company is not wedded to it. Jacobs said the company is more interested in a role in the Farenheit project. "Farenheit is going to be an important API over time, and there are going to be subcategories in that where we would like to play a role," he said.