Blog

The latest news from Google AI

Building Your Own Neural Machine Translation System in TensorFlow

Wednesday, July 12, 2017

Posted by Thang Luong, Research Scientist, and Eugene Brevdo, Staff Software Engineer, Google Brain TeamGoogle Translatedeep neural networks to capture sentence meaningsOpenNMTtf-seq2seqNeural Machine Translation (NMT) tutorialTensorFlowWMT’14TensorFlow 1.2

Easily read and preprocess dynamically sized input sequences using the new input pipeline in tf.contrib.data.

Use padded batching and sequence length bucketing to improve training and inference speeds.

Train seq2seq models using popular architectures and training schedules, including several types of attention and scheduled sampling.

Perform inference in seq2seq models using in-graph beam search.

Optimize seq2seq models for multi-GPU settings.

GitHubCore contributorsAcknowledgementsGoogle Brain teamSequence to sequence learning with neural networksIlya Sutskever, Oriol Vinyals, and Quoc V. Le. NIPS, 2014.Learning phrase representations using RNN encoder-decoder for statistical machine translationKyunghyun Cho, Bart Van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. EMNLP 2014.Neural machine translation by jointly learning to align and translateDzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. ICLR, 2015. Effective approaches to attention-based neural machine translationMinh-Thang Luong, Hieu Pham, and Christopher D Manning. EMNLP, 2015. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine TranslationYonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean. Technical Report, 2016.