Scalable and Efficient Mining of Association Rules in Distributed Database Environments

International Journal of Computer Science (IJCS) Published by SK Research Group of Companies (SKRGC)

Abstract

The growing volume of data in distributed databases has led to a need for more efficient methods to mine association rules. Traditional centralized techniques often struggle with issues related to scalability, performance and resource management when applied to distributed environments. This paper presents a new framework designed to improve scalability and efficiency in association rule mining for distributed databases. Our approach leverages distributed computing, optimized data partitioning, and parallel processing to reduce computational overhead and enhance mining performance. We introduce a two-phase algorithm that integrates local pattern discovery with global rule aggregation, minimizing communication costs while maintaining high accuracy. Experiments conducted on real-world datasets show that our method outperforms existing techniques in terms of execution time, scalability, and resource utilization. This research provides a foundation for future studies in distributed data mining and offers valuable insights for implementing association rule mining in large-scale systems.

References

1.Agrawal, R., &Srikant, R. (1994). Fast algorithms for mining association rules. Proceedings of the 20th International 
Conference on Very Large Data Bases (VLDB).

2.Han, J., Pei, J., & Yin, Y. (2000). Mining frequent patterns without candidate generation. ACM SIGMOD Record.

3.Dean, J., &Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM.

4.Zaharia, M., et al. (2016). Apache Spark: A unified engine for big data processing. Communications of the ACM.
 

Keywords

Association Rule Mining, Distributed Databases, Scalability, Parallel Processing, Data Partitioning, Big Data Analytics