An Automated Clustering Pipeline for Document Clustering over Distributed Web Proxies

International Journal of Computer Science and Technology Vol. 3, No. 6, June 2001 | pp. 41–48

DOI: 10.46882/2001/IJCST/000030

Research Article

Title: An Automated Clustering Pipeline for Document Clustering over Distributed Web Proxies

Names of Authors: Robert Evans¹, Yuki Tanaka²

Authors’ Affiliations: ¹Department of Software Engineering, University of Toronto, Canada; ²Graduate School of Information Science, Tokyo Institute of Technology, Japan.

Abstract:
The growth of textual data distributed across remote network caching proxies requires scalable clustering frameworks capable of grouping documents without high data relocation overheads. Executing text analysis over distributed nodes requires resilient ingestion architectures and fast categorization engines. This paper presents an integrated text clustering pipeline utilizing decentralized text preprocessing modules and an optimized k-means partitioning algorithm. The pipeline groups text elements across multi-node clusters, handling up to 15,000 files per second. We verified the architecture using a dataset of 5 million web records across a distributed proxy network. The processing pipeline achieved a classification accuracy of 89.4% while maintaining sub-second text categorization latency from ingestion to database storage. We introduced a customized token filtering module that handles syntax variations, increasing clustering purity metrics by 7.3%. Comparative benchmarks confirm that our distributed architecture outperforms traditional centralized storage workflows by a factor of 12, offering enterprises an efficient tool for distributed document management.

Keywords: Distributed Computing, Document Clustering, Ingestion Pipelines, Cluster Analysis, Performance Optimization, Text Mining.

Manuscript Timeline: Received: March 18, 2001; Revised: April 29, 2001; Accepted: May 15, 2001; Published: June 25, 2001.

Citation: Evans, R., & Tanaka, Y. (2001). An Automated Clustering Pipeline for Document Clustering over Distributed Web Proxies. International Journal of Computer Science and Technology, 3(6), 41–48.

Subscribe to read the full article: https://internationalscholarsjournals.org/subscribe-to-read