Designing a universal remote control for the ubiquitous computing environment

This is an abridged version of a paper that will be presented at the IEEE International Conference in Consumer Electronics, which opens in Los Angeles.

In the near future, there will be hundreds of networked appliances in homes and offices. What will be the user interface for these appliances? Does the average consumer want more infrared remote controls in their home? Is it necessary to alter the user interface according to the user?

Several design requirements must be met for an interface to work in an ubiquitous computing environment, specifically, it must provide explicit and easy selection of appliances, provision of an appropriate user interface for each appliance,support for multi-user operation and user identification and realization of differentiated service for each user.

There are several middleware architectures for ubiquitous computing environment such as Jini and UpnP. However, they do not satisfy the first requirement since their directory service based approach does not allow users to choose appliances intuitively. Infrared remote controllers are more usable in terms of intuitive manipulation. On the other hand, user interface of infrared remote controllers are usually predetermined and hard to be customized. Therefore it does not satisfy the second requirement.

A conventional laser pointer interaction system allows the user to explicitly choose an appliance with a laser pointer and to use variable user interfaces. However, since it utilizes image processing to recognize the laser spot, this system allows only one user to control appliances at any one time. Hence, it does not satisfy the third and fourth requirements.

In order to satisfy all the above requirements, we have developed a new universal remote control device that provides consumers with easy device selection and customized user interfaces - in terms of both appliances and users- while preventing leakage of personal information. Called the Smart Baton System, it is distinct from other laser-pointer-based manipulation techniques in that the user can download an appropriate user interface to their handheld devices via wireless network, as well as select the appliances with a laser pointer. Moreover, by modulating user's ID (network ID and port number of the users' device) onto the laser beam, the target appliances are able to recognize multiple users' operations so that it can provide differentiated services to different users.

The system consists of smart batons, SmartBaton-capable appliances and CA (certificate authority). Essentially, a smart baton is a handheld device such as a PDA equipped with a laser pointer, and is used to control appliances. A smartbaton-capable appliance, which is controlled by users, has a laser receiver and network connectivity. A CA is used to authenticate and identify users and appliances. When a user points at an appliance with a smart baton laser pointer, the user's ID travels to the appliance through the laser beam. Then, the appliance detects the beam to receive the information from its laser receiver, identifies the user's smart baton network ID and establishes a network connection to the smart baton. After that, an authentication process follows and the user's identity is proven. In this way, the appliance can provide different user interfaces and services to respective users. For example, the system can prevent children from turning on the TV at night without their parent's permission.

In the ubiquitous computing environment, appliances can exploit users' information to provide differentiated service. It follows that users are required to provide appliances with their private ID. In such a situation, users may worry about leakage of their private ID. We have addressed this concern by using two communication channels - a laser beam and a network connection- that have different characteristics between users and appliances where their information is transferred.

Secure solutions

The device selection phase is vulnerable to malicious appliances: if someone sets up a fake appliance which collects information transmitted through a laser beam, users' information encoded onto the beam can be stolen. To prevent such attempts, a laser beam conveys only non-confidential information in the system. It conveys the IP address of a smart baton, TCP port number, and randomly generated session identifier. TCP port number and the session identifier are one-time value. Hence, it is only the IP address of a smart baton that might be leaked.

The latter phase is vulnerable to wire-tapping, but it is fairly easy to avoid. For example, in our implementation, all communications on the network utilizes the Secure Socket Layer (SSL); it is not trivial for someone listening on the network to steal users' private information. This SSL based implementation also gives the system resistance to tampering because of the nature of SSL. Furthermore, appliances can be authenticated, which avoids appliance impersonation attack on the network.

Smart Baton System lets the user download an appropriate interface to a handheld device via the wireless network and select the appliances with a laser pointer. The system provides differentiated services to different users. Source: Shibaura Institute of Technology

For this implementation, we used COMPAQ iPAQ Pocket PC H3660 equipped with a custom laser transmitter for the smart baton, and a laptop PC equipped with a laser receiver as a SmartBaton-capable appliance. The laser transmitter consists of visible laser device and a micro controller (PIC16F628, Microchip Inc.) controlled through iPAQ's serial interface. The laser transmitter sends data via visible laser speeds. The laser receiver consists of a photo detector (solar cell), signal processing circuit, and a micro controller (PIC16F877, Microchip Inc.), and sends received data to the laptop through the serial interface.

The Session Keeper takes charge of authentication. The Device Control Server provides user interfaces and services to users. Both are written in C++. User interfaces are implemented in HTML. When an appliance serves a user, the Device Control Server sends a URL of a user interface to the smart baton. The URL refers to a web server running on the appliance. On the smart baton, a web browser is invoked and loaded for the URL. The user is then able to use the appliance through the Web browser.