iDMS: An Index-Based Framework for Tracking Distributed Multidimensional Data Streams

Histograms are a compact and effective way to summarize large datasets, representing data distribution by partitioning it into blocks or buckets for visualization and analysis. They find widespread use in diverse fields, from image processing to statistical analysis and database optimization. This paper focuses on merging and maintaining a centralized histogram for distributed summaries in multidimensional spaces. The proposed framework employs a dynamic index data structure to efficiently approximate data distribution from continuous distributed data streams. It offers a simple and implementable solution, gener-ating an up-to-date histogram for aggregated multidimensional data summaries while supporting both centralized and local histograms. Experimental results demonstrate its performance in parallel and distributed settings, considering communication cost, error rate, and practicality.