Context-Aware Document Summarization Architecture Using Transformer-Based Attention Weighting

International Journal of Computer Science and Technology | Vol. 4, No. 11, November 2023 | pp. 1–8

DOI: 10.46882/2023/IJCST/000226

Article Type: Original Research Paper

Title: Context-Aware Document Summarization Architecture Using Transformer-Based Attention Weighting

Names of Authors: Anna Kowalski¹, Hans-Jürgen Integrated²

Authors’ Affiliations: ¹Faculty of Electronics and Information Technology, Warsaw University of Technology, Warsaw, Poland; ²Institute for Cognitive Systems Research, University of Stuttgart, Stuttgart, Germany

Abstract: Automated document summarization frameworks frequently generate disconnected output strings or omit critical context when processing long technical texts. This paper introduces an abstractive document summarization model that uses a context-aware transformer architecture. The framework features a targeted attention-weighting layer that highlights domain-specific technical terminology and structural document headings. This layer prevents the model from dropping high-impact semantic connections across paragraphs. We integrated a pointer-generator network layer to copy unique named entities and numerical metrics accurately from source text fields into the summaries. The neural model was trained and validated using standard CNN/DailyMail and arXiv technical text corpora. The validation results show a ROUGE-L score improvement of 4.35 points over baseline text-summarization models. Human evaluation tracking confirmed that the generated summaries maintained high context consistency, minimizing factual hallucination rates to less than 1.8% across tested articles.

Keywords: Natural Language Processing, Document Summarization, Transformers, Attention Mechanism, Factual Consistency, Text Mining

Manuscript Timeline: Received: March 15, 2023; Revised: May 08, 2023; Accepted: June 22, 2023; Published: November 15, 2023

Citation: Kowalski, A., & Integrated, H. -J. (2023). Context-Aware Document Summarization Architecture Using Transformer-Based Attention Weighting. International Journal of Computer Science and Technology, 4(11), 1–8. DOI: 10.46882/2023/IJCST/000226

Subscribe to read the full article: https://internationalscholarsjournals.org/subscribe-to-read