Causal Intervention for Abstractive Related Work Generation
Liu, Jiachang, Zhang, Qi, Shi, Chongyang, Naseem, Usman, Wang, Shoujin, Hu, Liang, and Tsang, Ivor W. (2023) Causal Intervention for Abstractive Related Work Generation. In: Findings of the Association for Computational Linguistics: EMNLP 2023. pp. 2148-2159. From: EMNLP 2023: Conference on Empirical Methods in Natural Language Processing, 6-10 December 2023, Singapore.
|
PDF (Published Version)
- Published Version
Available under License Creative Commons Attribution. Download (1MB) | Preview |
Abstract
Abstractive related work generation has attracted increasing attention in generating coherent related work that helps readers grasp the current research. However, most existing models ignore the inherent causality during related work generation, leading to spurious correlations which downgrade the models' generation quality and generalizability. In this study, we argue that causal intervention can address such limitations and improve the quality and coherence of generated related work. To this end, we propose a novel Causal Intervention Module for Related Work Generation (CaM) to effectively capture causalities in the generation process. Specifically, we first model the relations among the sentence order, document (reference) correlations, and transitional content in related work generation using a causal graph. Then, to implement causal interventions and mitigate the negative impact of spurious correlations, we use do-calculus to derive ordinary conditional probabilities and identify causal effects through CaM. Finally, we subtly fuse CaM with Transformer to obtain an end-to-end related work generation framework. Extensive experiments on two real-world datasets show that CaM can effectively promote the model to learn causal relations and thus produce related work of higher quality and coherence.
Item ID: | 82150 |
---|---|
Item Type: | Conference Item (Research - E1) |
ISBN: | 9798891760615 |
Copyright Information: | Creative Commons 4.0 BY (Attribution) license |
Date Deposited: | 11 Mar 2024 23:43 |
FoR Codes: | 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460208 Natural language processing @ 100% |
SEO Codes: | 22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220403 Artificial intelligence @ 100% |
Downloads: |
Total: 56 Last 12 Months: 26 |
More Statistics |