Kasthuriarachchy, Buddhika, Chetty, Madhu, Shatte, Adrian, and Walls, Darren (2023) Meaning-Sensitive Text Data Augmentation with Intelligent Masking. ACM Transactions on Intelligent Systems and Technology, 14 (6). 104.

PDF (Published Version) - Published Version Restricted to Repository staff only

Abstract

With the recent popularity of applying large-scale deep neural network-based models for natural language processing (NLP), attention to develop methods for text data augmentation is at its peak, since the limited size of training data tends to significantly affect the accuracy of these models. To this end, we propose a novel text data augmentation technique called Intelligent Masking with Optimal Substitutions Text Data Augmentation (IMOSA). IMOSA, developed for labelled sentences, can identify the most favourable sentences and locate the appropriate word combinations in a particular sentence to replace and generate synthetic sentences with a meaning closer to the original sentence, while also significantly increasing the diversity of the dataset. We demonstrate that the proposed technique notably improves the performance of classifiers based on attention-based transformer models through the extensive experiments for five different text classification tasks which are performed under the low data regime in a context-aware NLP setting. The analysis clearly shows that IMOSA effectively generates more sentences using favourable original examples and completely ignores undesirable examples. Furthermore, the experiments carried out confirm IMOSA’s ability to add diversity to the augmented dataset using multiple distinct masking patterns against the same original sentence, which remarkably adds variety to the training dataset. IMOSA consistently outperforms the two key masked language model-based text data augmentation techniques, and demonstrates a robust performance against the critical challenging NLP tasks.

Item ID:	81594
Item Type:	Article (Research - C1)
ISSN:	2157-6904
Keywords:	Computing methodologies; Natural language processing; Text data augmentation; masked language model; IMOSA
Copyright Information:	© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Date Deposited:	20 Feb 2024 01:51
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460208 Natural language processing @ 50% 46 INFORMATION AND COMPUTING SCIENCES > 4611 Machine learning > 461104 Neural networks @ 50%
SEO Codes:	22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220403 Artificial intelligence @ 100%
Downloads:	Total: 1
	More Statistics