Debugging and Optimizing Distributed Systems

It may seem, at first, that a discussion on distributed systems has limited relevance to the day-to-day work of embedded developers. After all, distributed systems are often envisioned as large computing clusters or massive parallel-processing supercomputers—systems that, to all outward appearances, have little in common with a typical embedded project. But, in reality, much of the research done to advance the art of building distributed systems and to improve the tools used in their development can be applied to a variety of embedded systems, both large or small.

This paper is in four parts. First, it discusses why even single-node systems based on multiple, discrete software components can be characterized, and debugged, as distributed systems. Next, it looks at the benefits of such component-based designs, such as greater scalability and fault isolation. Then, it examines the challenges in creating any distributed system; for instance, deciding on a communication mechanism and gaining visibility into how the system behaves as a whole. And last, it explores development techniques and frameworks that help "narrow the scope" so that conventional tools, such application debuggers and profilers, can be used to understand the root causes of complex distributed behavior.