Sophisticated security protocols need to keep up wth complex Internet

Transmitting sensitive data over non-secret channels has always required encryption technologies to ensure that the data arrives without exposure to eavesdroppers. The Internet has made it possible to transmit vast volumes of data more rapidly and cheaply and to a wider audience than ever before. At the same time, strong encryption makes it possible to send data securely, to digitally sign it, to prove it was sent or received, and to guarantee its integrity.

But bulk transmission of data in a commercially secure yet reasonably fast way requires encryption technology that is both secure and fast. However, more powerful encryption algorithms and longer secret keys are slower computationally, while faster encryption is inevitably less secure. This is a natural consequence of the nature of encryption — the speed of encryption is proportional to the speed of breaking the code by doing a brute force search.

Thus, developers are in a quandary when choosing encryption technologies, needing to balance quality of service with security. There are two solutions to this performance versus security problem: hierarchical encryption and data fragmentation with signal diversity. The solutions do not require new encryption algorithms, but rather they change the way we use them.

A hierarchical approach to key exchange means that simple and relatively weak encryption and keys are used to encrypt small chunks of data, for example 10 seconds of video. Each chunk has its own key. New keys for this bottom level encryption are exchanged using a slightly stronger encryption, for example, whole-video keys could govern the exchange of the 10-second chunk keys.

At a higher level, there could be weekly keys, securing the exchange of whole-video keys, and at a yet higher level, a subscriber key could govern the exchange of weekly keys. At higher levels, the encryption is stronger but is used less frequently, so the overall computational cost is low.

Two factors govern the strength of the encryption technology used to secure some data. First, the value of each encrypted item determines the strength of the encryption algorithm used to secure it. The second factor: the duration of secrecy required dictates the lowest cost of breaking the encryption algorithm.

Any security solution should be either to expensive or should deliver the decrypted data too slowly for the eavesdropper to use. So, it is necessary to choose encryption algorithms and key lengths suitable to the material being ciphered.

For example in a networked video stream, the entire video will have a larger value than small sections of the video. An eavesdropper will put in more effort to break a cipher that hides an entire video than merely a few parts of it. Similarly, an eavesdropper would put more effort again into breaking a cipher that governed the issue of all videos to a given subscriber, and more yet again to breaking into the security surrounding the database of all videos or of all subscribers.

Hierarchical encryption applies the principle of cryptographic strength appropriate to the value and duration of secrecy of the data. It breaks data into small chunks, such as breaking a video stream into 10-second excerpts. It encrypts each chunk with a different key. This is the lowest level of encryption, and because the value of one chunk is small, it is where we use the fastest and weakest level encryption.

At this lowest level, the encryption is applied to every chunk of data. It is also the only encryption that is applied to the actual transmitted data (for example, the video). However, there are a substantial number of keys that an eavesdropper would require in order to decipher the entire data file. For example a 90-minute video broken into ten-second chunks would be encrypted with 540 chunk keys. If a single chunk key was compromised, then all that would be exposed is a single ten-second excerpt of video.

The distribution of chunk keys between sending and receiving hosts is managed by a second level of encryption. The value of the data encrypted at this level is higher, as it secures all chunk keys. Because of the higher value of the data, a stronger encryption is used, but the performance penalty is minimized since the encryption and decryption are performed less frequently.

Only the chunk key is encrypted and only when it changes, for instance, every ten seconds. If a second-level key was compromised, all chunk keys would be compromised until the second-level key itself was changed. In the video example, the second level key could be changed on a per-video basis, with a new "video" key for every video requested by the subscriber. Compromising one video key would mean that the chunk keys for the video would be easily recovered by an eavesdropper, but that the video key would become worthless at the completion of viewing the video.

The third-level key governs the distribution of second-level keys. Distribution of second-level keys will depend on the application, for example, the number of video keys would be the same as the number of different videos requested by a given subscriber. In our example this would be the "subscriber key" that secures all the subscriber's video keys. If a single third-level key was compromised, all videos requested by the subscriber would be compromised, hence the need for stronger encryption at this level.

There is inevitably a top level of encryption, which may be the third level. It governs all primary transactions, such as requesting and paying for individual videos,changing account details, and so on. It should be a form of public-key encryption as this allows the subscriber to identify themselves securely.

In a the three-level video stream, the lowest level of encryption, the chunk level, will almost always be performedusing a classical or secret-key encryption technique, as these are the fastest algorithms for both encryption and decryption. Similarly, the top level of encryption should use a strong public-key encryption with a large key, as the data value at this level is very high.

Intermediate levels of encryption should be chosen to complement the combined strength of the lower levels. In every case, the encryption chosen should make the effort required to illicitly decipher the communications more than a suspected eavesdropper would be willing to invest for the return.

Dispelling eavesdroppers

The effort required for an eavesdropper to decipher all chunk keys should be roughly similar to the effort required to decipher the next level key, and should cost more than the data's potential benefit to the eavesdropper. The same principle applies to second and higher-level keys. The effort required to discover all second-level keys should be similar to that required for the single higher-level key, and so on for all subsequent levels.

At no stage does the subscriber need to know the individual keys used in this process, except perhaps the top-level key which will change infrequently. The lower level keys are exchanged between the sending and receiving host machines, and the user need not be aware of any encryption taking place at all.

The actual number of encryption levels depends on the application data and the requirements of the vendors. For example, video transmission could use four levels of encryption, with the second level of keys changing for each entire video as before, then the third level changed weekly, and the top level per subscriber.

As another example, the Pretty Good Privacy (PGP) mail encryption tool likewise follows a two-level encryption technology. It was developed in the 1990s as a compromise between the speed of secret-key ciphers and the key ownership characteristics of the slower public-key ciphers. A message is encrypted with a secret-key cipher, then the key to the cipher is itself encrypted with the receiver's public key and appended to the message which is then transmitted.

Hierarchical encryption generalizes and improves on the two-level PGP encryption process, as it intentionally tailors the strength of the encryption at any level to the value and lifespan of the data being encrypted.

But while encryption technologies offer good protection against eavesdropping, there are other ways to improve security without much performance degradation. If security of transmission must be better than hierarchical encryption alone, then one complementary solution is a combination of non-symbolic fragmentation and signal diversity.

Non-symbolic fragmentation is the breaking up of data into fragments which are usually — but not necessarily — different in size to the size of the base unit of data (the "symbol") of the application. For example, ASCII data is represented in 8-bit symbols and Unicode is represented in 16-bit symbols. Breaking up data in this way ensures that whole symbols are spread across more than one fragment. Fragments are then disordered and/or divided into one or more fragment streams, usually with no two fragments adjacent in the fragment stream if they had been adjacent in the original data.

Network and path diversity break up a file or data stream into fragments which are then sent over many different channels, either in the same network or different networks. For instance, a message could be transmitted partly over the phone network and partly via satellite. TCP/IP does a similar thing in sending packets over different paths, doing so for load-balancing purposes and is invisible to the end application.

Network and path diversity deliberately introduce the same principle as a secure communications mechanism — an eavesdropper would need to intercept not just one transmission path but all paths. Sub-symbolic fragmentation of data is also introduced to further confuse any intercepted stream of data. This technology can be used with or without encryption and still remains secure.