Large scale inference and tomography for network monitoring and diagnosis
Today's Internet is a massive, distributed network which continues to explode in size as e-commerce and related activities grow. The heterogeneous and largely unregulated structure of the Internet renders tasks such as dynamic routing, optimized service provision, service level verification, and detection of anamolous/malicious behavior increasingly challenging tasks. The problem is compounded by the fact that one cannot rely on the cooperation of individual servers and routers to aid in the collection of network traffic measurements vital for these tasks. In many ways, network monitoring and inference problems bear a strong resemblance to other ``inverse problems'' in which key aspects of a system are not directly observable. Familiar signal processing problems such as tomographic image reconstruction, pattern recognition, system identification, and array processing all have interesting interpretations in the networking context. This article introduces the new field of large-scale network inference, a field which we believe will benefit greatly from the wealth of signal processing research and algorithms.