Google AI Blog: 2017

Evaluation of Speech for the Google Assistant

Thursday, December 21, 2017

Posted by Enrique Alfonseca, Staff Research Scientist, Google AssistantSearch Quality Rating Guidelinespublishing some of the first Google Assistant guidelines
Creating the Guidelinesweather this weekend

explicit linguistic knowledge and deep learning solutions

Information Satisfaction: the content of the answer should meet the information needs of the user.

Length: when a displayed answer is too long, users can quickly scan it visually and locate the relevant information. For voice answers, that is not possible. It is much more important to ensure that we provide a helpful amount of information, hopefully not too much or too little. Some of our previous work is currently in use for identifying the most relevant fragments of answers.

Formulation: it is much easier to understand a badly formulated written answer than an ungrammatical spoken answer, so more care has to be placed in ensuring grammatical correctness.

Elocution: spoken answers must have proper pronunciation and prosody. Improvements in text-to-speech generation, such as WaveNet and Tacotron 2, are quickly reducing the gap with human performance.

here

Tacotron 2: Generating Human-like Speech from Text

Tuesday, December 19, 2017

Posted by Jonathan Shen and Ruoming Pang, Software Engineers, on behalf of the Google Brain and Machine Perception TeamsTacotronWaveNetTacotron 2Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram PredictionsWaveNet

A detailed look at Tacotron 2's model architecture. The lower half of the image describes the sequence-to-sequence model that maps a sequence of letters to a spectrogram. For technical details, please refer to the paper.

Tacotron 2 audio samplesdecorummerlotAcknowledgementsJonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, RJ Skerry-Ryan, Rif A. Saurous, Yannis Agiomyrgiannakis, Yonghui Wu, Sound Understanding team, TTS Research team, and TensorFlow team.

Introducing NIMA: Neural Image Assessment

Monday, December 18, 2017

Posted by Hossein Talebi, Software Engineer and Peyman Milanfar Research Scientist, Machine Perceptionconvolutional neural networksaddress the subjective nature of image qualityNIMA: Neural Image Assessmentobject recognitionBackgroundPSNRSSIMImageNetNIMApaperAVAphotography contests

Ranking some examples labelled with the “landscape” tag from AVA dataset using NIMA. Predicted NIMA (and ground truth) scores are shown below each image.

TID2013

Ranking some examples from TID2013 dataset using NIMA. Predicted NIMA scores are shown below each image.

Perceptual Image Enhancementpaper

NIMA can be used as a training loss to enhance images. In this example, local tone and contrast of images is enhanced by training a deep CNN with NIMA as its loss. Test images are obtained from the MIT-Adobe FiveK dataset.

Looking Ahead

Improving End-to-End Models For Speech Recognition

Thursday, December 14, 2017

Posted by Tara N. Sainath, Research Scientist, Speech Team and Yonghui Wu, Software Engineer, Google Brain TeamphonemesState-of-the-art Speech Recognition With Sequence-to-Sequence Modelsword error ratelistenerxh^encattenderh^encxn0graphemewordpiecespeller

Components of the LAS End-to-End Model.

AcknowledgementsThis work was done as a strong collaborative effort between Google Brain and Speech teams. Contributors include Tara Sainath, Rohit Prabhavalkar, Bo Li, Kanishka Rao, Shankar Kumar, Shubham Toshniwal, Michiel Bacchiani and Johan Schalkwyk from the Speech team; as well as Yonghui Wu, Patrick Nguyen, Zhifeng Chen, Chung-cheng Chiu, Anjuli Kannan, Ron Weiss, Navdeep Jaitly, William Chan, Yu Zhang and Jan Chorowski from the Google Brain team. The work is described in more detail in papers [4-12].ReferencesLower Frame Rate Neural Network Acoustic ModelsListen, attend and spellA Comparison of Sequence-to-sequence Models for Speech RecognitionState-of-the-art Speech Recognition With Sequence-to-Sequence ModelsMinimum Word Error Rate Training for Attention-based Sequence-to-Sequence ModelsMulti-Dialect Speech Recognition With a Single Sequence-to-Sequence ModelEnd-to-End Multilingual Speech Recognition using Encoder-Decoder ModelsImproving the Performance of Online Neural Transducer ModelsMonotonic Chunkwise AttentionLearning Hard Alignments with Variational InferenceNo Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End ModelsAn Analysis of Incorporating an External Language Model into a Sequence-to-Sequence Model

A Summary of the First Conference on Robot Learning

Wednesday, December 13, 2017

Posted by Vincent Vanhoucke, Principal Scientist, Google Brain Team and Melanie Saldaña, Program Manager, University RelationsConference on Robot Learning (CoRL)74 original papers5 keynotes

Prof. Sergey Levine, CoRL 2017 co-chair, answering audience questions.

Sayna Ebrahimi (UC Berkeley) presenting her research.

conference websiteProfessor Aude BillardÉcole Polytechnique Fédérale de LausanneEidgenössische Technische Hochschule

Prof. Ken Goldberg, CoRL 2017 co-chair, and Jeffrey Mahler (UC Berkeley) during a break.

TFGAN: A Lightweight Library for Generative Adversarial Networks

Tuesday, December 12, 2017

Posted by Joel Shor, Senior Software Engineer, Machine Perception(Crossposted on the Google Open Source Blog)image compressiontext-to-speech systemsGenerative Adversarial Networksgenerating images from textsuperresolutionhelping robots learn to grasp

A video of a generator improving over time. It begins by producing random noise, and eventually learns to generate MNIST digits.

TFGANexamplestutorial

This demonstrates the effect of an adversarial loss on image compression. The top row shows image patches from the ImageNet dataset. The middle row shows the results of compressing and uncompressing an image through an image compression neural network trained on a traditional loss. The bottom row shows the results from a network trained with a traditional loss and an adversarial loss. The GAN-loss images are sharper and more detailed, even if they are less like the original.

Most neural text-to-speech (TTS) systems produce over-smoothed spectrograms. When applied to the Tacotron TTS system, a GAN can recreate some of the realistic-texture, which reduces artifacts in the resulting audio.

Introducing Appsperiments: Exploring the Potentials of Mobile Photography

Monday, December 11, 2017

Posted by Alex Kauffmann, Interaction Researcher, Google ResearchappsperimentsMotion Stillsobject recognitionperson segmentationStoryboard

AndroidSelfissimo!

iOSAndroidScrubbies

iOSAcknowledgementsThese appsperiments represent a collaboration across many teams at Google. We would like to thank the core contributors Andy Dahley, Ashley Ma, Dexter Allen, Ignacio Garcia Dorado, Madison Le, Mark Bowers, Pascal Getreuer, Robin Debreuil, Suhong Jin, and William Lindmeier. We also wish to give special thanks to Buck Bourdon, Hossein Talebi, Kanstantsin Sokal, Karthik Raveendran, Matthias Grundmann, Peyman Milanfar, Suril Shah, Tomas Izo, Tyler Mullen, and Zheng Sun.

Introducing a New Foveation Pipeline for Virtual/Mixed Reality

Tuesday, December 5, 2017

Posted by Behnam Bastani, Software Engineer Manager and Eric Turner, Software Engineer, DaydreamVirtual RealityMixed RealityresearchFoveated Renderingfovea centralissaccades,

The left image is rendered at full resolution. The right image uses two layers of foveation — one rendered at high resolution (inside the yellow region) and one at lower resolution (outside).

A smooth full rendering (image on the left). The image on the right shows temporal artifacts introduced by motion in foveated region.

Phase-Aligned Foveated RenderingConformal Foveated RenderingPhase-Aligned Renderingfrustums

Temporal artifacts in non-world aligned foveated rendered content (left) and the phase-aligned method (right).

Conformal Renderingrasterized

Comparison of traditional foveation (left) to conformal rendering (right), where content is rendered to a space matched to visual perception acuity and HMD lens characteristics. Both methods use the same number of total pixels.

Foveated Image Processing

Correction for head-mounted-display lens chromatic aberration in foveated space. Top image shows the conventional pipeline. The bottom image (in Green) shows the operation in the foveated space.

The left image shows reconstructed foveated content after lens distortion. The right image shows image difference when lens distortion correction is performed in a foveated manner. The right image shows that minimal error is introduced close to edges of frame buffer. These errors are imperceptible in an HMD.

Foveated Transmissionsystem-on-a-chip

Rather than streaming upscaled foveated content (left image), foveated transmission enables streaming content pre-reconstruction (right image) and reducing the number of bits transmitted.

Comparison of full integration of foveation and compression techniques (left) versus typical flickering artifacts that may be introduced by applying DSC to foveated content (right).

Toward a New PipelineAcknowledgementsWe would like to recognize the work done by the following collaborators:

Haomiao Jiang and Carlin Vieri on display compression and foveated transmission

Brian Funt and Sylvain Vignaud on the development of new foveated rendering algorithms

DeepVariant: Highly Accurate Genomes With Deep Neural Networks

Monday, December 4, 2017

Posted by Mark DePristo and Ryan Poplin, Google Brain Team(Crossposted on the Google Open Source Blog)Sanger sequencingmicroarray technologieshealthagricultureecologyGenome in a Bottle ConsortiumprecisionFDA

For any given location in the genome, there are multiple reads among the ~1 billion that include a base at that position. Each read is aligned to a reference, and then each of the bases in the read is compared to the base of the reference at that location. When a read includes a base that differs from the reference, it may indicate a variant (a difference in the true sequence), or it may be an error.

open source release of DeepVariantGoogle Brain teamVerily Life Sciencesexisting technology and expertise

Each of the four images above is a visualization of actual sequencer reads aligned to a reference genome. A key question is how to use the reads to determine whether there is a variant on both chromosomes, on just one chromosome, or on neither chromosome. There is more than one type of variant, with SNPs and insertions/deletions being the most common. A: a true SNP on one chromosome pair, B: a deletion on one chromosome, C: a deletion on both chromosomes, D: a false variant caused by errors. It's easy to see that these look quite distinct when visualized in this manner.

DeepVariantSNPprecisionFDA Truth Challenge

Google Cloud PlatformDeepVariant workflowsPipelines APIhealthcarescientific applications

Google at NIPS 2017

Sunday, December 3, 2017

Posted by Christian Howard, Editor-in-Chief, Research CommunicationsstConference on Neural Information Processing SystemsblueOrganizing CommitteeSamy BengioCorinna Cortes, Dale Schuurmans, Hugo LarochelleAfshin Rostamizadeh, Amir Globerson, Been Kim, D. Sculley, Dumitru Erhan, Gal Chechik, Hartmut Neven, Honglak Lee, Ian Goodfellow, Jasper Snoek, John Wright, Jon Shlens, Lihong Li, Maya Gupta, Moritz Hardt, Navdeep Jaitly, Ryan Adams, Sally Goldman, Sanjiv Kumar, Surya Ganguli, Tara Sainath, Umar Syed, Viren Jain, Vitaly KuznetsovInvited TalkPowering the next 100 yearsJohn PlattAccepted PapersA Meta-Learning Perspective on Cold-Start Recommendations for ItemsManasi Vartak, Hugo Larochelle, Arvind ThiagarajanAdaGAN: Boosting Generative ModelsIlya Tolstikhin, Sylvain Gelly, Olivier Bousquet, Carl-Johann Simon-Gabriel, Bernhard SchölkopfDeep Lattice Networks and Partial Monotonic FunctionsSeungil You, David Ding, Kevin Canini, Jan Pfeifer, Maya GuptaFrom which world is your graphCheng Li, Varun Kanade, Felix MF Wong, Zhenming LiuHiding Images in Plain Sight: Deep SteganographyShumeet BalujaImproved Graph Laplacian via Geometric Self-ConsistencyDominique Joncas, Marina Meila, James McQueenModel-Powered Conditional Independence TestRajat Sen, Ananda Theertha Suresh, Karthikeyan Shanmugam, Alexandros Dimakis, Sanjay ShakkottaiNonlinear random matrix theory for deep learningJeffrey Pennington, Pratik WorahResurrecting the sigmoid in deep learning through dynamical isometry: theory and practiceJeffrey Pennington, Samuel Schoenholz, Surya GanguliSGD Learns the Conjugate Kernel Class of the NetworkAmit DanielySVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and InterpretabilityMaithra Raghu, Justin Gilmer, Jason Yosinski, Jascha Sohl-DicksteinLearning Hierarchical Information Flow with Recurrent Neural ModulesDanijar Hafner, Alexander Irpan, James Davidson, Nicolas HeessOnline Learning with Transductive RegretScott Yang, Mehryar MohriAcceleration and Averaging in Stochastic Descent DynamicsWalid Krichene, Peter BartlettParameter-Free Online Learning via Model SelectionDylan J Foster, Satyen Kale, Mehryar Mohri, Karthik SridharanDynamic Routing Between CapsulesSara Sabour, Nicholas Frosst, Geoffrey E HintonModulating early visual processing by languageHarm de Vries, Florian Strub, Jeremie Mary, Hugo Larochelle, Olivier Pietquin, Aaron C CourvilleMarrNet: 3D Shape Reconstruction via 2.5D SketchesJiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, Bill Freeman, Josh TenenbaumAffinity Clustering: Hierarchical Clustering at ScaleMahsa Derakhshan, Soheil Behnezhad, Mohammadhossein Bateni, Vahab Mirrokni, MohammadTaghi Hajiaghayi, Silvio Lattanzi, Raimondas KiverisAsynchronous Parallel Coordinate Minimization for MAP InferenceOfer Meshi, Alexander SchwingCold-Start Reinforcement Learning with Softmax Policy GradientNan Ding, Radu SoricutFiltering Variational ObjectivesChris J Maddison, Dieterich Lawson, George Tucker, Mohammad Norouzi, Nicolas Heess, Andriy Mnih, Yee Whye Teh, Arnaud DoucetMulti-Armed Bandits with Metric Movement CostsTomer Koren, Roi Livni, Yishay MansourMultiscale Quantization for Fast Similarity SearchXiang Wu, Ruiqi Guo, Ananda Theertha Suresh, Sanjiv Kumar, Daniel Holtmann-Rice, David Simcha, Felix YuReducing Reparameterization Gradient VarianceAndrew Miller, Nicholas Foti, Alexander D'Amour, Ryan AdamsStatistical Cost SharingEric Balkanski, Umar Syed, Sergei VassilvitskiiThe Unreasonable Effectiveness of Structured Random Orthogonal EmbeddingsKrzysztof Choromanski, Mark Rowland, Adrian WellerValue Prediction NetworkJunhyuk Oh, Satinder Singh, Honglak LeeREBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelsGeorge Tucker, Andriy Mnih, Chris J Maddison, Dieterich Lawson, Jascha Sohl-DicksteinApproximation and Convergence Properties of Generative Adversarial LearningShuang Liu, Olivier Bousquet, Kamalika ChaudhuriAttention is All you NeedAshish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia PolosukhinPASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inferenceJonathan Huggins, Ryan Adams, Tamara BroderickRepeated Inverse Reinforcement LearningKareem Amin, Nan Jiang, Satinder SinghFair Clustering Through FairletsFlavio Chierichetti, Ravi Kumar, Silvio Lattanzi, Sergei VassilvitskiiAffine-Invariant Online Optimization and the Low-rank Experts ProblemTomer Koren, Roi LivniBatch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized ModelsSergey IoffeBridging the Gap Between Value and Policy Based Reinforcement LearningOfir Nachum, Mohammad Norouzi, Kelvin Xu, Dale SchuurmansDiscriminative State Space ModelsVitaly Kuznetsov, Mehryar MohriDynamic Revenue SharingSantiago Balseiro, Max Lin, Vahab Mirrokni, Renato Leme, Song ZuoMulti-view Matrix Factorization for Linear Dynamical System EstimationMahdi Karami, Martha White, Dale Schuurmans, Csaba SzepesvariOn Blackbox Backpropagation and Jacobian SensingKrzysztof Choromanski, Vikas SindhwaniOn the Consistency of Quick ShiftHeinrich JiangRevenue Optimization with Approximate Bid PredictionsAndres Munoz, Sergei VassilvitskiiShape and Material from SoundZhoutong Zhang, Qiujia Li, Zhengjia Huang, Jiajun Wu, Josh Tenenbaum, Bill FreemanLearning to See Physics via Visual De-animationJiajun Wu, Erika Lu, Pushmeet Kohli, Bill Freeman, Josh TenenbaumConference DemosElectronic Screen Protector with Efficient and Robust Mobile VisionHee Jung Ryu, Florian SchroffMagenta and deeplearn.js: Real-time Control of DeepGenerative Music Models in the BrowserCurtis Hawthorne, Ian Simon, Adam Roberts, Jesse Engel, Daniel Smilkov, Nikhil Thorat, Douglas EckWorkshops6th Workshop on Automated Knowledge Base ConstructionArvind NeelakantaJiazhong Nie, Ni LaoActing and Interacting in the Real World: Challenges in Robot LearningPierre SermanetAdvances in Approximate Bayesian InferenceMatthew D. HoffmanConversational AI - Today's Practice and Tomorrow's PotentialMatthew Henderson, Dilek Hakkani-TurLarry HeckExtreme Classification: Multi-class and Multi-label Learning in Extremely Large Label SpacesEd Chi, Mehryar MohriLearning in the Presence of Strategic BehaviorMehryar MohriAndres Munoz Medina, Sebastien Lahaie, Sergei Vassilvitskii, Balasubramanian SivanLearning on Distributions, Functions, Graphs and GroupsCorinna CortesMachine DeceptionIan GoodfellowJacob Buckman, Aurko Roy, Colin Raffel, Ian GoodfellowMachine Learning and Computer SecurityIan GoodfellowNicolas PapernotJacob Buckman, Aurko Roy, Colin Raffel, Ian GoodfellowMachine Learning for Creativity and DesignIan GoodfellowDoug Eck, David HaMachine Learning for Audio Signal Processing (ML4Audio)Aren Jansen, Manoj Plakal, Dan Ellis, Shawn Hershey, Channing Moore, Rif A. Saurous, Yuxuan Wang, RJ Skerry-Ryan, Ying Xiao, Daisy Stanton, Joel Shor, Eric Batternberg, Rob ClarkMachine Learning for Health (ML4H)Jasper Snoek, Alex WiltschkoFei-Fei LiNIPS Time Series Workshop 2017Vitaly KuznetsovBrendan JouOPT 2017: Optimization for Machine LearningSashank ReddiML Systems WorkshopRajat Monga, Alexander Mordvintsev, Chris Olah, Jeff DeanAlex Beutel, Tim Kraska, Ed H. Chi, D. Scully, Michael TerryAligned Artificial IntelligenceIan GoodfellowBayesian Deep LearningKevin MurphyNal Kalchbrenner, Matthew D. HoffmanBigNeuro 2017Viren JainCognitively Informed Artificial Intelligence: Insights From Natural IntelligenceJiazhong Nie, Ni LaoDeep Learning At Supercomputer ScaleErich Elsen, Zak Stone, Brennan Saeta, Danijar HaffnerDeep Learning: Bridging Theory and PracticeIan GoodfellowInterpreting, Explaining and Visualizing Deep LearningBeen Kim, Honglak LeePieter Kinderman, Sara Hooker, Dumitru Erhan, Been KimLearning Disentangled Features: from Perception to ControlHonglak LeeJasmine Hsu, Arkanath Pathak, Abhinav Gupta, James Davidson, Honglak LeeLearning with Limited Labeled Data: Weak Supervision and BeyondIan GoodfellowMachine Learning on the Phone and other Consumer DevicesRajat MongaHrishikesh AradhyeSuyog Gupta, Sujith RaviOptimal Transport and Machine LearningOlivier BousquetThe future of gradient-based machine learning software & techniquesAlex Wiltschko, Bart van MerriënboerWorkshop on Meta-LearningHugo LarochelleSamy BengioAliaksei Severyn, Sascha RotheSymposiumsDeep Reinforcement Learning SymposiumBenjamin Eysenbach, Shane Gu, Julian Ibarz, Sergey LevineInterpretable Machine LearningMinmin ChenMetalearningQuoc V LeCompetitionsAdversarial Attacks and DefencesAlexey Kurakin, Ian Goodfellow, Samy BengioCompetition IV: Classifying Clinically Actionable Genetic MutationsWendy KanTutorialFairness in Machine LearningSolon Barocas, Moritz Hardt

Blog