Deep Copycat Networks for Text-to-Text Generation

Julia Ive, Pranava Madhyastha, Lucia Specia


Abstract
Most text-to-text generation tasks, for example text summarisation and text simplification, require copying words from the input to the output. We introduce Copycat, a transformer-based pointer network for such tasks which obtains competitive results in abstractive text summarisation and generates more abstractive summaries. We propose a further extension of this architecture for automatic post-editing, where generation is conditioned over two inputs (source language and machine translation), and the model is capable of deciding where to copy information from. This approach achieves competitive performance when compared to state-of-the-art automated post-editing systems. More importantly, we show that it addresses a well-known limitation of automatic post-editing - overcorrecting translations - and that our novel mechanism for copying source language words improves the results.
Anthology ID:
D19-1318
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3227–3236
Language:
URL:
https://aclanthology.org/D19-1318
DOI:
10.18653/v1/D19-1318
Bibkey:
Cite (ACL):
Julia Ive, Pranava Madhyastha, and Lucia Specia. 2019. Deep Copycat Networks for Text-to-Text Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3227–3236, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Deep Copycat Networks for Text-to-Text Generation (Ive et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1318.pdf