How uClinux provides MMU-less processors with an alternative

How uClinux provides MMU-less processors with an alternative
Many software developers in recent years have turned to Linux as their operating system of choice. Until the advent of uClinux developers of smaller embedded systems, usually incorporating microprocessors with no memory management unit could not take advantage of Linux in their designs according to Michael Durrant and Michael Leslie of Lineo.

UClinux is a variant of mainstream Linux that runs on 'MMU-less' processor architectures. Component costs are of primary concern in embedded systems, which are typically required to be small and inexpensive. Microprocessors with on-chip memory management unit (MMU) hardware tend to be complex and expensive, and as such are not typically selected for small, simple embedded systems which do not require them.

Benefits of Linux

Using Linux in devices which require some intelligence is attractive for many reasons:

-It is a mature, robust operating system-It already supports a large number of devices, filesystems, and networking protocols -Bug fixes and new features are constantly being added, tested and refined by a large community of programmers and users -It gives everyone from developers to end users complete visibility of the source code -A large number of applications (such as GNU software) exist which require little to no porting effort -Linux's very low cost

Embedded systems running uClinux may be configured in many ways other than that of the familiar UNIX-like Linux distribution. Nevertheless, an example of a system running uClinux in this way will help to illustrate how it may be used.

Kernel/root filesystem

Lineo's uCdimm is a complete computer in an SO-DIMM form factor, built around a Motorola 68VZ328 'Dragonball' microcontroller, the latest processor in a family widely popularised by the 'Palm Pilot. It is equipped with 2M of flash memory, 8M of SDRAM, both synchronous and asynchronous serial ports, and an ethernet controller. There is a custom resident boot monitor on the device, which is capable of downloading a binary image to flash memory and executing it. The image that is downloaded consists of a uClinux kernel and root filesystem.

In UNIX terms, the kernel makes a block device out of the memory range where the root filesystem resides, and mounts this device as root. The root filesystem is in a read-only UNIX-like format called 'ROMFS'.

Since the Dragonball runs at 32MHz, the kernel and optionally user programs execute in-place in flash memory. Faster systems benefit from copying the kernel and root filesystem into RAM and executing there.

Other embedded systems may be inherently network-based, so a kernel in flash memory might mount a root filesystem being served via network file system (NFS). An even more network-centric device might request its kernel image and root filesystems via dynamic host configuration protocol (DHCP) and bootp. Note that drivers for things like IDE and SCSI disk, CD, and floppy support are all still present in the uClinux kernel.

User space

The contents of the root filesystem vary more dramatically between embedded systems using uClinux than between Linux workstations. The uClinux distribution contains a root filesystem which implements a small UNIX-like server, with a console on the serial port, a telnet daemon, a web server, NFS client support, and a selection of common UNIX tools. A system such as an MPEG layer 3 compressed audio CD player might not even have a console. The kernel might contain only support for a CD drive, parallel I/O, and an audio DAC. User space might consist only of an interface program to drive buttons and LEDs, to control the CD, and which could invoke one other program; an MPEG audio player. Such an application specific system would obviously require much less memory than the full-fledged uClinux distribution as it is shipped.

Development under uClinux

Developing software for uClinux systems typically involves a cross-compiler toolchain built from the familiar GNU compiler tools. Software that builds under GNU C Compiler (GCC) for x86 architectures, for example, often builds without modification on any uClinux target.Debugging a target via GNU debugger (GDB) presents a debugging interface common to all the platforms supported by GDB.

The debugging interface to a uClinux kernel on a target depends on debugging support for that target. If the target processor has hardware support for debugging, such as IEEE's JTAG or Motorola's BDM, GDB may connect non-intrusively to the target to debug the kernel. If the processor lacks such support, a GDB 'stub' may be incorporated into the kernel. GDB communicates with the stub via a serial port, or via Ethernet.

The C library used in uClinux, uClibc, is a smaller implementation than those which ship with most modern Linux distributions. The library has been designed to provide most of the calls that UNIX-like C programs will use. If an application requires a feature that is not implemented in uClibc, the feature may be added to uClibc, it may me linked in as a separate library, or it may be added to the application itself.

Differences between uClinux and Linux

Considering that the absence of MMU support in uClinux constitutes a fundamental difference from mainstream Linux, surprisingly little kernel and user space software is affected. Developers familiar with Linux will notice little difference working under uClinux. Embedded systems developers will already be familiar with some of the issues peculiar to uClinux. Two differences between mainstream Linux and uClinux are a consequence of the removal of MMU support from uClinux. The lack of both memory protection and of a virtual memory model are of importance to a developer working in either kernel or user space. Certain system calls to the kernel are also affected.

Memory protection

One consequence of operating without memory protection is that an invalid pointer reference by even an unprivileged process may trigger an address error, and potentially corrupt or even shut down the system. Obviously code running on such a system must be programmed carefully and tested diligently to ensure robustness and security.

There are three primary consequences of running Linux without virtual memory. One is that processes which are loaded by the kernel must be able to run independently of their position in memory. One way to achieve this is to "fix up" address references in a program once it is loaded into RAM. The other is to generate code that uses only relative addressing (referred to as PIC, or Position Independent Code) - uClinux supports both of these methods.

Another consequence is that memory allocation and deallocation occurs within a flat memory model. Very dynamic memory allocation can result in fragmentation which can starve the system. One way to improve the robustness of applications that perform dynamic memory allocation is to replace malloc() calls with requests from a preallocated buffer pool.

Since virtual memory is not used in uClinux, swapping pages in and out of memory is not implemented, since it cannot be guaranteed that the pages would be loaded to the same location in RAM. In embedded systems it is also unlikely that it would be acceptable to suspend an application in order to use more RAM than is physically available.

Changes to the interface

The lack of memory management hardware on uClinux target processors has meant that some changes needed to be made to the Linux system interface. Perhaps the greatest difference is the absence of the fork() and brk() system calls.

A call to fork() clones a process to create a child. Under Linux, fork() is implemented using copy-on-write pages. Without an MMU, uClinux cannot completely and reliably clone a process, nor does it have access to copy-on-write.

uClinux implements vfork() in order to compensate for the lack of fork(). When a parent process calls vfork() to create a child, both processes share all their memory space including the stack. vfork() then suspends the parent's execution until the child process either calls exit() or execve().

Note that multitasking is not otherwise affected. It does, however, mean that older-style network daemons that make extensive use of fork() must be modified. Since child processes run in the same address space as their parents, the behaviour of both processes may require modification in particular situations.

Many modern programs rely on child processes to perform basic tasks, allowing the system to maintain an interactive 'feel' even if the processing load is quite heavy. Such programs may require substantial reworking to perform the same task under uClinux. If a key application depends heavily on such structuring, then it may be necessary to either re-create the application, or an MMU-enabled processor may also be needed.

A hypothetical, simple network daemon, hyped, will illustrate the use of fork().hyped always listens on a well-known network port (or socket) for connections from a network client. When the client connects, hyped gives it new connection information (a new socket number) and calls fork(). The child process then accepts the client's reconnection to the new socket, freeing the parent to listen for new connections.

uClinux has neither an autogrow stack nor brk() and so user space programs must use the mmap() command to allocate memory. For convenience, our C library implements malloc() as a wrapper to mmap(). There is a compile-time option to set the stack size of a program.

Anatomy of the uClinux kernel

This section describes the changes that were made to the Linux kernel to allow it to run on MMU-less processors.

The architecture-generic memory management subsystem was modified to remove reliance on MMU hardware by providing basic memory management functions within the kernel software itself.

For those who are familiar with uClinux, this is the role of the directory /mmnommu derived from and replacing the directory /mm. Several subsystems needed to be modified, added, removed, or rewritten. Kernel and user memory allocation and deallocation routines had to be reimplemented.

Support for transparent swapping/paging was removed. Program loaders which support position independent code (PIC) were added. A new binary object code format, named 'flat' was created, which supports PIC and which has a very compact header. Other program loaders, such as that for ELF, were modified to support other formats which, instead of using PIC, use absolute references which it is the responsibility of the kernel to 'fix up' at run time.

Each method has advantages and disadvantages. Traditional PIC is quick and compact but has a size restriction on some architectures. For example, the 16-bit relative jump in Motorola 68k architectures limits PIC programs to 32K. The runtime fix-up technique removes this size restriction, but incurs overhead when the program is loaded by the kernel.

Porting uClinux to new platforms

The task of adding support for a new CPU architecture in uClinux is similar to doing so in Linux proper. Fortunately, there is a great deal of code in Linux that can be ported with minor adaptations and reused in uClinux. Machine dependent startup code and header files already exist in Linux for MMU versions of processors in the ARM, Motorola 68k, MIPS, SPARC and other families. This code may be adapted to support non-MMU versions of these processors in uClinux.

Driver code which already exists in Linux is often easily portable to run under uClinux. Issues in porting such code may involve endian issues or memory handling code which assumes the presence of MMU support.

Numerous enhancements are in the works for uClinux. The diversity of the innovations that mainstream Linux receives from the community pave a good path for the development of uClinux. The uClinux developer community is very active; enhancements and innovations are frequently made.

Linux is now a platform for hard real-time application development (that is, applications with deterministic latency under varying processor loads). The Linux kernel scheduler already provides non-deterministic, or 'soft', real-time, and systems such as real-time application interface (RTAI) upgrade the Linux kernel to provide hard (deterministic) real-time support. Real-time applications in Linux have access to the extensive resources of the Linux kernel without sacrificing hard real-time performance. Efforts are underway to provide the RTAI subsystem for use on various MMU-less processors.

uClinux 2.4, with support for Motorola Dragonball and Coldfire, was released in January of 2001. Other ports have been made or are being planned to the uClinux 2.4 tree, which is based on Linux 2.4. but enhancements are also still being made to the uClinux 2.0 tree. uClinux 2.4 will give developers access to many of the new features added to Linux since 2.0, including support for USB, IEEE Firewire, IrDA, and new networking features such as bandwidth allocation, (a.k.a. QoS: Quality of Service) IP Tables, and IPv6.

Since uClinux is Open Source, development effort spent on uClinux will never be lost. Engineering professionals world-wide, are using uClinux to create commercial products and a significant portion of their work is contributed back to the open source community.

Why create an MMU-less Linux? As early as 1997, Jeff Dionne, Michael Durrant, and others discussed the possibility of implementing Linux on MMU-less processors to act as a low cost network controller driving data communications between Ethernet and Microwave communication systems. However, it was the collaboration of Kenneth Albanowski and Jeff Dionne that resulted in the world's first release. This early uClinux implementation was deployed into a SCADA controller and publicly released into the open source community as an alternative OS for the Palm Pilot (Feb 1998).

Jeff Dionne and Michael Durrant from Lineo Canada (formerly Rt-Control) later designed and built a line of embedded controllers know as uCsimm and uCdimm (see photo of uCdimm) taking advantage of uClinux's compact code size. Meanwhile, Greg Ungerer, Chief Scientist, Lineo Australia (formerly MoretonBay) ported uClinux onto the popular Motorola ColdFire platform and designed several VPN (Virtual Private Network) Internet appliances including Lineo's NetTEL and SecureEdge routers (see photo of SecureEdge).

The uClinux kernel has been deployed on several other CPU architectures, and platforms including the AXIS Network Camera from Axis Communications (Sweden), and the Voice over IP (VoIP) telephone from Aplio SA (France). Notable contributions to uClinux have been made by engineers at Lineo, Aplio, Axis, and from individuals in the open source community. These contributions are reflected on Lineo's open source web site (opensource.lineo.com).

References 'Running Linux on low cost, low power, MMU-less processors', Michael Durrant, Lineo, Inc. 'Building Low Cost, Embedded, Network Appliance with Linux', Greg Ungerer, Lineo, Inc. 'Embedded Coldfire-Taking Linux On-Board', Nigel Dick, Motorola Ltd 'When hard real-time goes soft', D. Jeff Dionne, Lineo, Inc.

Michael Durrant is director of engineering at Lineo and Michael Leslie is a senior software developer at Lineo.