Transformer Models for Text Summarization: A Comparative Study of BART, BERT, and RoBERTa
Transformer Models for Text Summarization: A Comparative Study of BART, BERT, and RoBERTa
用于文本摘要的 Transformer 模型：BART、BERT 和 RoBERTa 的比较研究
Abstract: Text summarization refers to the task of condensing a document into a shorter version while preserving its key information. Automatic text summarization (ATS), driven by advancements in natural language processing (NLP), has developed rapidly in recent years. 摘要： 文本摘要是指在保留文档关键信息的前提下，将其压缩为简短版本的过程。在自然语言处理（NLP）进步的推动下，自动文本摘要（ATS）近年来发展迅速。
ATS methods are commonly categorized by input type (such as single-document or multi-document summarization) and by output type (extractive, abstractive, and hybrid). ATS 方法通常根据输入类型（如单文档或多文档摘要）和输出类型（抽取式、生成式和混合式）进行分类。
This article presents a focused review of modern summarization techniques with an emphasis on transformer based models and large language models (LLMs), specifically BERT, RoBERTa and BART. It examines their architectures, pretraining strategies, and their suitability for extractive and abstractive summarization tasks. 本文重点综述了现代摘要技术，特别强调了基于 Transformer 的模型和大语言模型（LLMs），具体包括 BERT、RoBERTa 和 BART。文章探讨了它们的架构、预训练策略，以及它们在抽取式和生成式摘要任务中的适用性。
Publication Details:
- Authors: Daisy Aptovska, Vinayak Elangovan
- Journal: International Journal of Artificial Intelligence and Applications (IJAIA), Vol.17, No.3, May 2026
- arXiv ID: 2608.19200
- Date: 2 Jun 2026
出版详情：
- 作者： Daisy Aptovska, Vinayak Elangovan
- 期刊： 《人工智能与应用国际期刊》(IJAIA)，第17卷，第3期，2026年5月
- arXiv ID： 2608.19200
- 日期： 2026年6月2日