Refining Source Representations with Relation Networks for Neural Machine Translation

Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu


Abstract
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neural network structure, and disregarding relationship between source words during encoding step. Whereas in practice, the former information and relationship are often useful in current step. We target on solving these problems and thus introduce relation networks to learn better representations of the source. The relation networks are able to facilitate memorization capability of recurrent neural network via associating source words with each other, this would also help retain their relationships. Then the source representations and all the relations are fed into the attention component together while decoding, with the main encoder-decoder framework unchanged. Experiments on several datasets show that our method can improve the translation performance significantly over the conventional encoder-decoder model and even outperform the approach involving supervised syntactic knowledge.
Anthology ID:
C18-1110
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1292–1303
Language:
URL:
https://aclanthology.org/C18-1110
DOI:
Bibkey:
Cite (ACL):
Wen Zhang, Jiawei Hu, Yang Feng, and Qun Liu. 2018. Refining Source Representations with Relation Networks for Neural Machine Translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1292–1303, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Refining Source Representations with Relation Networks for Neural Machine Translation (Zhang et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1110.pdf