Google AI Blog

Duality — A New Approach to Reinforcement Learning

Wednesday, July 8, 2020

Posted by Ofir Nachum and Bo Dai, Research Scientists, Google ResearchReinforcement learningQ-learningactor-criticconstraint-satisfaction problemmathematical issuesReinforcement Learning via Fenchel-Rockafellar Dualityconvex dualityA Duality-Based Solutionconvex regularizer f-divergenceunconstrainedoff-policyoffline

do notExperimental Evaluation

A plot of the average reward achieved by an agent using the duality-based approach (blue) compared to an agent using standard actor-critic (orange). In addition to being more mathematically principled, our approach also yields better practical results.

Conclusionwithout constraintspolicy optimizationpolicy evaluationimitation learning

Google at ACL 2020

Monday, July 6, 2020

Posted by Cat Armato and Emily Knapp, Program ManagersACL 2020Diamond Level sponsor of ACL 2020Google virtual boothboldedCommitteesVinodkumar Prabhakaran Sushant Kafle Kristina ToutanovaYi Luan Anders Søgaard, Ankur Parikh, Annie Louis, Bhuvana Ramabhadran, Christo Kirov, Daniel Cer, Dipanjan Das, Diyi Yang, Emily Pitler, Eunsol Choi, George Foster, Idan Szpektor, Jacob Eisenstein, Jason Baldridge, Jun Suzuki, Kenton Lee, Luheng He, Marius Pasca, Ming-Wei Chang, Sebastian Gehrmann, Shashi Narayan, Slav Petrov, Vinodkumar Prabhakaran, Waleed Ammar, William CohenLong PapersCross-modal Language Generation using Pivot Stabilization for Web-scale Language CoverageAshish V. Thapliyal, Radu Soricut Automatic Detection of Generated Text is Easiest when Humans are FooledDaphne Ippolito, Daniel Duckworth, Chris Callison-Burch, Douglas Eck On Faithfulness and Factuality in Abstractive SummarizationJoshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan McDonald MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited DevicesZhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, Denny Zhou BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby StepsWang Zhu, Hexiang Hu, Jiacheng Chen, Zhiwei Deng, Vihan Jain, Eugene Ie, Fei ShaDynamic Programming Encoding for Subword Segmentation in Neural Machine TranslationXuanli He, Gholamreza Haffari, Mohammad Norouzi GoEmotions: A Dataset of Fine-Grained EmotionsDorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, Sujith Ravi TaPas: Weakly Supervised Table Parsing via Pre-trainingsee blog post)Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, Julian EisenschlosToxicity Detection: Does Context Really Matter?John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, Ion Androutsopoulos (Re)construing Meaning in NLPSean Trott, Tiago Timponi Torrent, Nancy Chang, Nathan SchneiderPretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language ModelsDan Iter, Kelvin Guu, Larry Lansing, Dan Jurafsky Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question AnsweringHao Cheng, Ming-Wei Chang, Kenton Lee, Kristina Toutanova AdvAug: Robust Adversarial Augmentation for Neural Machine TranslationYong Cheng, Lu Jiang, Wolfgang Macherey, Jacob Eisenstein Named Entity Recognition as Dependency ParsingJuntao Yu, Bernd Bohnet, Massimo Poesio Cross-modal Coherence Modeling for Caption GenerationMalihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone Representation Learning for Information Extraction from Form-like Documents(see blog post)Bodhisattwa Prasad Majumder, Navneet Potti, Sandeep Tata, James Bradley Wendt, Qi Zhao, Marc NajorkLow-Dimensional Hyperbolic Knowledge Graph EmbeddingsInes Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, Christopher Ré What Question Answering can Learn from Trivia NerdsJordan Boyd-Graber, Benjamin Börschinger Learning a Multi-Domain Curriculum for Neural Machine Translation(see blog post)Wei Wang, Ye Tian, Jiquan Ngiam, Yinfei Yang, Isaac Caswell, Zarana Parekh Translationese as a Language in "Multilingual" NMTParker Riley, Isaac Caswell, Markus Freitag, David Grangier Mapping Natural Language Instructions to Mobile UI Action SequencesYang Li, Jiacong He, Xin Zhou, Yuan Zhang, Jason Baldridge BLEURT: Learning Robust Metrics for Text Generation(see blog post)Thibault Sellam, Dipanjan Das, Ankur Parikh Exploring Unexplored Generalization Challenges for Cross-Database Semantic ParsingAlane Suhr, Ming-Wei Chang, Peter Shaw, Kenton Lee Frugal Paradigm CompletionAlexander Erdmann, Tom Kenter, Markus Becker, Christian Schallhart Short PapersReverse Engineering Configurations of Neural Text Generation ModelsYi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, Andrew TomkinsSyntactic Data Augmentation Increases Robustness to Inference HeuristicsJunghyun Min, R. Thomas McCoy, Dipanjan Das, Emily Pitler, Tal Linzen Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine TranslationAditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan, Yonghui WuSocial Biases in NLP Models as Barriers for Persons with DisabilitiesBen Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, Stephen Denuyl Toward Better Storylines with Sentence-Level Language ModelsDaphne Ippolito, David Grangier, Douglas Eck, Chris Callison-Burch TACL PapersTYDI QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages(see blog post)Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, Jennimaria PalomakiPhonotactic Complexity and Its Trade-offsTiago Pimentel, Brian Roark, Ryan CotterellDemosMultilingual Universal Sentence Encoder for Semantic Retrievalsee blog postYinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, Ray KurzweilWorkshopsIWPT - The 16th International Conference on Parsing TechnologiesYuji Matsumoto, Stephan Oepen, Kenji Sagae, Anders Søgaard, Weiwei Sun and Reut TsarfatyALVR - Workshop on Advances in Language and Vision ResearchXin Wang, Jesse Thomason, Ronghang Hu, Xinlei Chen, Peter Anderson, Qi Wu, Asli Celikyilmaz, Jason Baldridge and William Yang WangWNGT - The 4th Workshop on Neural Generation and Translation Alexandra Birch, Graham Neubig, Andrew Finch, Hiroaki Hayashi, Kenneth Heafield, Ioannis Konstas, Yusuke Oda and Xian LiNLPMC - NLP for Medical ConversationsParminder Bhatia, Chaitanya Shivade, Mona Diab, Byron Wallace, Rashmi Gangadharaiah, Nan Du, Izhak Shafran and Steven LinAutoSimTrans - The 1st Workshop on Automatic Simultaneous TranslationHua Wu, Colin Cherry, James Cross, Liang Huang, Zhongjun He, Mark Liberman and Yang LiuTutorialsInterpretability and Analysis in Neural NLP (cutting-edge)Yonatan Belinkov, Sebastian Gehrmann, Ellie PavlickCommonsense Reasoning for Natural Language Processing (Introductory)Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi, Dan Roth

SmartReply for YouTube Creators

Wednesday, July 1, 2020

Posted by Rami Al-Rfou, Research Scientist, Google ResearchSmartReplyGmail launchAndroid MessagesAndroid WearPlay Developer ConsoleMLKitTFLite

Deep RetrievalSmartReply for Inboxrecurrent neural networkwe foundmaximum inner product searchexpand SmartReply to GmailBeyond Wordsour recent workresearchTransformerself-attention layersWaveNetcontrastive objectiveaverage pooling

A dual encoder network that maximizes the mutual information between the comments and their replies through a contrastive objective. Each encoder is fed a sequence of bytes and is implemented as a computationally efficient dilated transformer network.

A Model to Learn Them AllThis is a great video,Thanks!

A 2D projection of the model encodings when presented with a hypothetical comment and a small list of potential replies. The neighborhood surrounding English comments (black color) consists of appropriate replies in English and their counterparts in Spanish and Arabic. Note that the network learned to align English replies with their translations without access to any parallel corpus.

When to Suggest?ConclusionAcknowledgementsSmartReply for YouTube creators was developed by Golnaz Farhadi, Ezequiel Baril, Cheng Lee, Claire Yuan, Coty Morrison‎, Joe Simunic‎, Rachel Bransom‎, Rajvi Mehta, Jorge Gonzalez‎, Mark Williams, Uma Roy and many more. We are grateful for the leadership support from Nikhil Dandekar, Eileen Long, Siobhan Quinn, Yun-hsuan Sung, Rachel Bernstein, and Ray Kurzweil.

SpineNet: A Novel Architecture for Object Detection Discovered with Neural Architecture Search

Tuesday, June 30, 2020

Posted by Xianzhi Du, Software Engineer and Jaeyoun Kim, Technical Program Manager, Google ResearchConvolutional neural networks created for image tasks typically encode an input image into a sequence of intermediate features that capture the semantics of an image (from local to global), where each subsequent layer has a lower spatial dimension. However, this scale-decreased model may not be able to deliver strong features for multi-scale visual recognition tasks where recognition and localization are both important (e.g., object detection and segmentation). Several works including FPN and DeepLabv3+ propose multi-scale encoder-decoder architectures to address this issue, where a scale-decreased network (e.g., a ResNet) is taken as the encoder (commonly referred to as a backbone model). A decoder network is then applied to the backbone to recover the spatial information.
While this architecture has yielded improved success for image recognition and localization tasks, it still relies on a scale-decreased backbone that throws away spatial information by down-sampling, which the decoder then must attempt to recover. What if one were to design an alternate backbone model that avoids this loss of spatial information, and is thus inherently well-suited for simultaneous image recognition and localization?CVPR 2020SpineNet: Learning Scale-Permuted Backbone for Recognition and Localizationscale-permuted modelneural architecture searchTensorflow TPU GitHub repositoryTensorFlow Model Garden GitHub repository

A scale-decreased backbone is shown on the left and a scale-permuted backbone is shown on the right. Each rectangle represents a building block. Colors and shapes represent different spatial resolutions and feature dimensions. Arrows represent connections among building blocks.

Design of SpineNet ArchitectureCOCO dataset

Scale permutations: The orderings of network building blocks are important because each block can only be built from those that already exist (i.e., with a “lower ordering”). We define the search space of scale permutations by rearranging intermediate and output blocks, respectively.

Cross-scale connections: We define two input connections for each block in the search space. The parent blocks can be any block with a lower ordering or a block from the stem network.

Block adjustments (optional): We allow the block to adjust its scale level and type.

The architecture search process from a scale-decreased backbone to a scale-permuted backbone.

ResNet-50ResNet-50-FPNaverage precisionFLOPsresidual blockbottleneck blockResNet

The architecture comparison between a ResNet backbone (left) and the SpineNet backbone (right) derived from it using NAS.

PerformanceCOCOiNaturalist

Performance comparisons of SpineNet models and ResNet-FPN models adopting the RetinaNet detection framework on COCO bounding box detection.

Performance comparisons of SpineNet models and ResNet models on ImageNet classification and iNaturalist fine-grained image classification.

ConclusionAcknowledgementsSpecial thanks to the co-authors of the paper: Tsung-Yi Lin, Pengchong Jin, Golnaz Ghiasi, Mingxing Tan, Yin Cui, Quoc V. Le, and Xiaodan Song. We also would like to acknowledge Yeqing Li, Youlong Cheng, Jing Li, Jianwei Xie, Russell Power, Hongkun Yu, Chad Richards, Liang-Chieh Chen, Anelia Angelova, and the larger Google Brain Team for their help.

Leveraging Temporal Context for Object Detection

Friday, June 26, 2020

Posted by Sara Beery, Student Researcher, and Jonathan Huang, Research Scientist, Google Researchmountain pass road conditionsecosystem phenologyleading to poor generalization to novel deploymentsLILA BCWildlife Insights

What’s in this image? Objects in images from static cameras can be very challenging to detect and categorize. Here, a foggy morning has made it very difficult to see a herd of wildebeest walking along the crest of a hill. [Image from Snapshot Serengeti]

Context R-CNN: Long Term Temporal Context for Per-Camera Object DetectionFaster R-CNNTF Object Detection API

Here, we can see how additional examples from the same scene help experts determine that the object is an animal and not background. Context such as the shape & size of the object, its attachment to a herd, and habitual grazing at certain times of day help determine that the species is a wildebeest. Useful examples occur throughout the month.

The Context R-CNN ModelFaster R-CNNmemory bankattention

High-level architecture diagram, showing how Context R-CNN incorporates long-term context within the Faster R-CNN model architecture.

similarity-based attentionmemory bankMM

Context R-CNN is able to leverage context (spanning up to 1 month) to correctly categorize the challenging wildebeest example we saw above. The green values are the corresponding attention weights for each boxed object.

Compared to a Faster R-CNN baseline (left), Context R-CNN (right) is able to capture challenging objects such as an elephant occluded by a tree, two poorly-lit impala, and a vervet monkey leaving the frame. [Images from Snapshot Serengeti]

ResultsSnapshot SerengetiCaltech Camera Trapsmean average precisionS3D

Comparison to a single frame Faster R-CNN baseline, showing both mean average precision (mAP) and average recall (AR) detection metrics.

Ongoing and Future WorkWildlife InsightsiWildCamCVPR Fine-Grained Visual Recognition WorkshopAcknowledgementsThis post reflects the work of the authors as well as the following group of core contributors: Vivek Rathod, Guanhang Wu, Ronny Votel. We are also grateful to Zhichao Lu, David Ross, Tanya Birch and the Wildlife Insights AI team, and Pietro Perona and the Caltech Computational Vision Lab.

Sensing Force-Based Gestures on the Pixel 4

Wednesday, June 24, 2020

Posted by Philip Quinn and Wenxin Feng, Research Scientists, Android UXsoftfirmrecent update to the Pixel 4Touch Sensor Technology and Finger Biomechanicsdrive electrodesense electrodedielectriccell

Left: A finger interacts with a touch sensor cell by ‘stealing’ charge from the projected field around two electrodes. Right: A capacitive touch sensor is constructed from rows and columns of electrodes, separated by a dielectric. The electrodes overlap at cells, where capacitance is measured.

Slowed touch sensor recordings of a user tapping (left), pressing (middle), and scrolling (right).

per se

Touch sensor signals are saturated around the centre of the finger’s contact, but fall off at the edges. This allows us to sense small deformations in the finger’s contact shape caused by changes in the finger’s force.

Machine Learning for Touch Interactionpress gestureconvolutionalrecurrentTensorFlow Lite

An overview of the classification model’s architecture.

User Experience Integrationpress

A user long pressing (left) and firmly pressing (right) on a launcher icon.

GestureDetectorViewMotionEvent classificationAcknowledgementsThis project is a collaborative effort between the Android UX, Pixel software, and Android framework teams.

Presenting a Challenge and Workshop in Efficient Open-Domain Question Answering

Tuesday, June 23, 2020

Posted by Eunsol Choi, Visiting Faculty Researcher and Tom Kwiatkowski, Research Scientist, Google Researchknowledge graphWikipediaT5EfficientQA competition and workshopNeurIPS 2020Princeton UniversityUniversity of Washingtonquestion answering

An illustration of how the memory budget changes as a neural network and retrieval corpus grow and shrink. It is possible that successful systems will also use other resources such as a knowledge graph.

Competition OverviewNatural Questions datasetcompetition track at NeurIPS 20202017 NeurIPS Human-Computer competitionParticipationcompetition siteAcknowledgementsAdam Roberts, Colin Raffel, Chris Alberti, Jordan Boyd-Graber, Jennimaria Palomaki, Kenton Lee, Kelvin Guu, Michael CollinsSewon Min Hannaneh HajishirzDanqi Chen

RepNet: Counting Repetitions in Videos

Monday, June 22, 2020

Posted by Debidatta Dwibedi, Research Scientist, Robotics at GoogleCounting Out Time: Class Agnostic Video Repetition Counting in the WildRepNetour previous work, whichdatasetColab notebookRepNetIn the pastResNetResNetimagevideotemporal self-similarity matrix

Demonstration of how the TSM processes images of the Earth’s day-night cycle.

Transformers

Overview of the RepNet model.

Temporal Self-Similarity Matrix

Jumping Jacks (constant period; video from Kinetics), Bouncing ball (decreasing period; Kinetics), Mixing concrete (aperiodic segments present in video; PERTUBE dataset).

Data

Our synthetic data generation pipeline that produces videos with repetitions from any video.

camera motion augmentationaffine motion

Left: An example of a synthetic repeating video generated from a random video. Right: An example of a video with camera motion augmentation, which is tougher for the model, but results in better generalization to real repeating videos (both from Kinetics).

EvaluationKinetics datasetCountixApplications

RepNet can count repeated activities from a range of domains, such as slicing onions (left; video from Kinetics dataset), Earth’s diurnal cycle (middle; Himawari satellite data), or even a cheetah in motion (right; video from imgur.com).

Predicted heart rates: 45 bpm (left) and 75 bpm (right). True heart rates 46-50 bpm and 78-79 bpm, respectively. RepNet’s prediction of the heart rate across different devices is encouragingly close to the rate measured by the device. (Source for left and right)

In this video, we see RepNet counting accelerating cellular oscillations observed under a laser microscope even though it has never seen such a video during training, (from Nature article).

Left: Person performing a “mountain climber” exercise. Right: The 1D projection of the RepNet embeddings using principal component analysis, capturing the moment that the person changes their speed during the exercise. (Video from Kinetics)

ReleaseannotationsColab notebookAcknowledgementsThis is joint work with Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. Special thanks to Tom Small for designing the visual explanation of TSM. The authors thank Anelia Angelova, Relja Arandjelović, Sourish Chaudhuri, Aishwarya Gomatam, Meghana Thotakuri, and Vincent Vanhoucke for their help with this project.

Improving Speech Representations and Personalized Models Using Self-Supervision

Thursday, June 18, 2020

Posted by Joel Shor, Software Engineer and Oran Lang, Software Engineer, Google Research, Israelautomatic speech recognitionBERTALBERTInception layersSimCLR

Bottom:A large speech dataset is used to train a model, which is then rolled out to other environments. Top Left: On-device personalization — personalized, on-device models combine security and privacy. Top Middle: Small model on embeddings — general-use representations transform high-dimensional, few-example datasets to a lower dimension without sacrificing accuracy; smaller models train faster and are regularized. Top Right: Full model fine-tuning — large datasets can use the embedding model as pre-training to improve performance

T5Visual Task Adaptation BenchmarkTowards Learning a Universal Non-Semantic Representation of SpeechThese datasetsTensorFlow DatasetsTRIpLet Loss networkopen-source the codeA New Benchmark for Speech Embeddings

Datasets for downstream benchmark tasks. *VoxCeleb results in our study were computed using a subset of the dataset that was filtered according to internal policy.

kkinternet of thingsadded the six datasets in our benchmark to TensorFlow Datasetsopen sourced the evaluation frameworkTRILL: A New State of the Art in Non-semantic Speech Classificationpersonalizing speech recognizersvoice imitation text-to-speech from few samplesAudioSetpreviousworkBERT

TRILL loss: Embeddings from the same audio are closer in embedding space than embeddings from different audio.

MobileNetResNet50Benchmark ResultsOpenSMILEstrong baselinesy

Effect of model on accuracy.

Interspeech 2020 Computational Paralinguistics Challenge (ComParE)baseline modelSummaryGitHubTensorFlow DatasetsAI HubAcknowledgementsThe core team behind this work includes Joel Shor, Aren Jansen, Ronnie Maor, Oran Lang, Omry Tuval, Felix de Chaumont Quitry, Marco Tagliasacchi, Ira Shavitt, Dotan Emanuel, and Yinnon Haviv. We'd also like to thank Avinatan Hassidim and Yossi Matias for technical guidance.

Blog