Jump to content

Neural machine translation: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Line 9: Line 9:


==History==
==History==
Deep learning applications appeared first in speech recognition in the 1990s. The first scientific paper on using neural networks in machine translation appeared in 2014, followed by a lot of advances in the following few years. (Large-vocabulary NMT, application to Image captioning, Subword-NMT, Multilingual NMT, Multi-Source NMT, Character-dec NMT, Zero-Resource NMT, Google, Fully Character-NMT, Zero-Shot NMT in 2017) In 2015 there was the first appearance of a NMT system in a public machine translation competition (OpenNMT'15). WMT'15 also for the first time had a NMT contender; the following year it already had 90% of NMT systems among its winners.<ref name="WMT16"/>
Deep learning applications appeared first in speech recognition in the 1990s. The first scientific paper on using neural networks in machine translation appeared in 2014, followed by a lot of advances in the following few years. (Large-vocabulary NMT, application to Image captioning, Subword-NMT, Multilingual NMT, Multi-Source NMT, Character-dec NMT, Zero-Resource NMT, Google, Fully Character-NMT, Zero-Shot NMT in 2017) In 2015 there was the first appearance of a NMT system in a public machine translation competition (OpenMT'15). WMT'15 also for the first time had a NMT contender; the following year it already had 90% of NMT systems among its winners.<ref name="WMT16"/>


==Workings==
==Workings==

Revision as of 12:49, 5 March 2019

Neural machine translation (NMT) is an approach to machine translation that uses a large artificial neural network to predict the likelihood of a sequence of words, typically modeling entire sentences in a single integrated model.

Deep neural machine translation is an extension of neural machine translation. Both use a large neural network with the difference that deep neural machine translation processes multiple neural network layers instead of just one.[1]

Properties

They require only a fraction of the memory needed by traditional statistical machine translation (SMT) models. Furthermore, unlike conventional translation systems, all parts of the neural translation model are trained jointly (end-to-end) to maximize the translation performance.[2][3][4]

History

Deep learning applications appeared first in speech recognition in the 1990s. The first scientific paper on using neural networks in machine translation appeared in 2014, followed by a lot of advances in the following few years. (Large-vocabulary NMT, application to Image captioning, Subword-NMT, Multilingual NMT, Multi-Source NMT, Character-dec NMT, Zero-Resource NMT, Google, Fully Character-NMT, Zero-Shot NMT in 2017) In 2015 there was the first appearance of a NMT system in a public machine translation competition (OpenMT'15). WMT'15 also for the first time had a NMT contender; the following year it already had 90% of NMT systems among its winners.[5]

Workings

NMT departs from phrase-based statistical approaches that use separately engineered subcomponents.[6] Neural machine translation (NMT) is not a drastic step beyond what has been traditionally done in statistical machine translation (SMT). Its main departure is the use of vector representations ("embeddings", "continuous space representations") for words and internal states. The structure of the models is simpler than phrase-based models. There is no separate language model, translation model, and reordering model, but just a single sequence model that predicts one word at a time. However, this sequence prediction is conditioned on the entire source sentence and the entire already produced target sequence.[7]

NMT models use deep learning and representation learning.

The word sequence modeling was at first typically done using a recurrent neural network (RNN). A bidirectional recurrent neural network, known as an encoder, is used by the neural network to encode a source sentence for a second RNN, known as a decoder, that is used to predict words in the target language.[8]

Convolutional Neural Networks (Convnets) are in principle somewhat better for long continuous sequences, but were initially not used due to several weaknesses that were successfully compensated for by 2017 by using so-called "attention"-based approaches.[9][10]

Usage

By 2016, most of the best MT systems were using neural networks.[5] Google, Microsoft and Yandex[11] translation services now use NMT. Google uses Google Neural Machine Translation (GNMT) in preference to its previous statistical methods.[12] Microsoft uses a similar technology for its speech translations (including Microsoft Translator live and Skype Translator).[13] An open source neural machine translation system, OpenNMT, has been released by the Harvard NLP group.[14]

References

  1. ^ "Deep Neural Machine Translation". Omniscien Technologies. Retrieved 2017-11-08.
  2. ^ Kalchbrenner, Nal; Blunsom, Philip (2013). "Recurrent Continuous Translation Models". Proceedings of the Association for Computational Linguistics.
  3. ^ Sutskever, Ilya; Vinyals, Oriol; Le, Quoc Viet (2014). "Sequence to sequence learning with neural networks". arXiv:1409.3215 [cs.CL].
  4. ^ Kyunghyun Cho; Bart van Merrienboer; Dzmitry Bahdanau; Yoshua Bengio (3 September 2014). "On the Properties of Neural Machine Translation: Encoder–Decoder Approaches". arXiv:1409.1259 [cs.CL].
  5. ^ a b Bojar, Ondrej; Chatterjee, Rajen; Federmann, Christian; Graham, Yvette; Haddow, Barry; Huck, Matthias; Yepes, Antonio Jimeno; Koehn, Philipp; Logacheva, Varvara; Monz, Christof; Negri, Matteo; Névéol, Aurélie; Neves, Mariana; Popel, Martin; Post, Matt; Rubino, Raphael; Scarton, Carolina; Specia, Lucia; Turchi, Marco; Verspoor, Karin; Zampieri, Marcos (2016). "Findings of the 2016 Conference on Machine Translation" (PDF). ACL 2016 First Conference on Machine Translation (WMT16). The Association for Computational Linguistics: 131–198.
  6. ^ Wołk, Krzysztof; Marasek, Krzysztof (2015). "Neural-based Machine Translation for Medical Text Domain. Based on European Medicines Agency Leaflet Texts". Procedia Computer Science. 64 (64): 2–9. doi:10.1016/j.procs.2015.08.456.
  7. ^ Philipp Koehn (2016-11-30). "The State of Neural Machine Translation (NMT)". Omniscien Technologies. Retrieved 2017-11-08.
  8. ^ Dzmitry Bahdanau; Cho Kyunghyun; Yoshua Bengio (2014). "Neural Machine Translation by Jointly Learning to Align and Translate". arXiv:1409.0473 [cs.CL].
  9. ^ Bahdanau, Dzmitry; Cho, Kyunghyun; Bengio, Yoshua (2014-09-01). "Neural Machine Translation by Jointly Learning to Align and Translate". arXiv:1409.0473 [cs.CL].
  10. ^ Coldewey, Devin (2017-08-29). "DeepL schools other online translators with clever machine learning". TechCrunch. Retrieved 2018-01-27.
  11. ^ "Yandex — Company blog — One model is better than two. Yandex.Translate launches a hybrid machine translation system". Yandex. Retrieved 2018-01-27.
  12. ^ Lewis-Kraus, Gideon (December 14, 2016). "The Great A.I. Awakening". The New York Times. Retrieved 2016-12-21.
  13. ^ "Microsoft Translator launching Neural Network based translations for all its speech languages". Translator. Retrieved 2018-01-27.
  14. ^ "OpenNMT – Open-Source Neural Machine Translation". opennmt.net. Retrieved 2017-03-22.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Neural_machine_translation&oldid=886303080"