Google AI Blog: July 2020

Introducing the Model Card Toolkit for Easier Model Transparency Reporting

Wednesday, July 29, 2020

Posted by Huanming Fang and Hui Miao, Software Engineers, Google ResearchModel Cardslaunched Model Cards publicly MediaPipetheir GitHub repositoryModel Card ToolkitColab tutorialUCI Census Income datasetIntroducing the MCTJSON schemaML Metadata

one UI templateTensorFlow ExtendedGoogle Cloud Platform

Here is an example of the completed Model Card from the Colab tutorial, which leverages the MCT and the provided UI template.

Conclusionget started herethe Colab tutorialmodel-cards@google.comTensorFlow Responsible AI pageAcknowledgementsHuanming Fang, Hui Miao, Karan Shukla, Dan Nanas, Catherina Xu, Christina Greer, Neoklis Polyzotis, Tulsee Doshi, Tiffany Deng, Margaret Mitchell, Timnit Gebru, Andrew Zaldivar, Mahima Pushkarna, Meena Natarajan, Roy Kim, Parker Barnes, Tom Murray, Susanna Ricco, Lucy Vasserman, and Simone Wu

Announcing ScaNN: Efficient Vector Similarity Search

Tuesday, July 28, 2020

Posted by Philip Sun, Software Engineer, Google ResearchICML 2020Accelerating Large-Scale Inference with Anisotropic Vector Quantization,”vector similarity search libraryann-benchmarks.comThe Importance of Vector Similarity Search

The two-tower neural network model, illustrated above, is a specific type of embedding-based search where queries and database items are mapped to the embedding space by two respective neural networks. In this example the model responds to natural-language queries for a hypothetical literary database.

nearest neighbor searchinner productmaximum inner-product searchA New Quantization Approach for MIPSSeveral state-of-the-art solutionslearned quantizationcodebookxx̃higherx1x2c1c2x_ix̃_iqx̃_iqx_ix̃_iqx_iqx_iqx̃₁greaterqx̃2qx1lessqx2x1c1x2c2

The goal is to quantize each x_i to x̃_i = c₁ or x̃_i = c₂. Traditional quantization (left) results in the incorrect ordering of x₁ and x₂ for this query. Even though our approach (right) chooses centers farther away from the data points, this in fact leads to lower inner product error and higher accuracy.

c1x1c2c1x1x1c2x2anisotropic vector quantization

In the above diagrams, ellipses denote contours of equal loss. In anisotropic vector quantization, error parallel to the original data point x is penalized more.

Anisotropic Vector Quantization in ScaNNkglove-100-angular benchmarkann-benchmarks.com^*

Recall@k is a commonly used metric for nearest neighbor search accuracy, which measures the proportion of the true nearest k neighbors that are present in an algorithm’s returned k neighbors. ScaNN (upper purple line) consistently achieves superior performance across various points of the speed-accuracy trade-off.

GitHubConclusionstate-of-the-art performanceAcknowledgementsThis post reflects the work of the entire ScaNN team: David Simcha, Erik Lindgren, Felix Chern, Nathan Cordeiro, Ruiqi Guo, Sanjiv Kumar, and Zonglin Li. We’d also like to thank Dan Holtmann-Rice, Dave Dopson, and Felix Yu.
* ScaNN performs similarly well on the other datasets of ann-benchmarks.com, but the website currently shows outdated, lower numbers. See this pull request for more representative performance figures on other datasets. ↩

Improving Holistic Scene Understanding with Panoptic-DeepLab

Tuesday, July 21, 2020

Posted by Bowen Cheng, Student Researcher, and Liang-Chieh Chen, Research Scientist, Google Researchinstance segmentationsemantic segmentationsemantic segmentationinstancesPanoptic segmentationMask R-CNNPanoptic-DeepLab: a simple, fast and strong system for panoptic segmentationCVPR 2020DeepLabbounding box predictionsUPSNet

Panoptic segmentation results obtained by Panoptic-DeepLab. Left: Video frames used as input to the panoptic segmentation model. Right: Results overlaid on video frames. Each object instance has a unique label, e.g., car_1, car_2, etc.

Overview

Overview of Panoptic-DeepLab. Semantic segmentation associates pixels in the image with general classes, while the class-agnostic instance segmentation step identifies the pixels associated with an individual object, regardless of the class. Taken together one gets the final panoptic segmentation image.

Neural Network DesignImageNetatrous spatial pyramid poolingDeepLabdecoder modulesatrous convolutionDeepLabV3+ResultsCityscapesMapillary VistasCOCOCityscapesBest ResultBest PaperMost InnovativeMapillary Panoptic Segmentation trackICCV 2019 Joint COCO and Mapillary Recognition Challenge WorkshopCOCO

Accuracy (PQ) vs. Speed (GPU inference time) across three datasets.

ConclusionMask R-CNNAcknowledgementsWe would like to thank the support and valuable discussions with Maxwell D. Collins, Yukun Zhu, Ting Liu, Thomas S. Huang, Hartwig Adam, Florian Schroff as well as the Google Mobile Vision team.

Exploring Faster Screening with Fewer Tests via Bayesian Group Testing

Tuesday, July 14, 2020

Posted by Marco Cuturi and Jean-Philippe Vert, Research Scientists, Google Research, Brain TeamRobert Dorfmanpaper

Left: Sixteen individual tests are required to screen 16 people — only one person’s test is positive, while 15 return negative. Right: Following Dorfman’s procedure, samples are pooled into four groups of four individuals, and tests are executed on the pooled samples. Because only the second group tests positive, 12 individuals are cleared and only those four belonging to the positive group need to be retested. This approach requires only eight tests, instead of the 16 needed for an exhaustive testing campaign.

information theorycombinatoricscompressive sensingbinary splittingindividualinfection probabilitynoiseless noisyadaptivenon-adaptiveNoisy Adaptive Group Testing using Bayesian Sequential Experimental Designour GitHub repoNoisy and Adaptive Group Testing in a Non-Asymptotic RegimenkknsensitivityspecificityScreening More People with Fewer Tests Using Bayesian Optimal Experimental Design

Given a population of n people, an infection state is a binary vector of length n that describes who is infected (marked with a 1), and who is not (marked with a 0). At a certain time, a population is in a given state (most likely a few 1’s and mostly 0’s). The goal of group testing is to identify that state using as few tests as possible. Given a prior belief on the infection rate (the disease is rare) and test results observed so far (if any), we expect that only a small share of those infection states will be plausible. Rather than evaluating the plausibility of all 2ⁿ possible states (an extremely large number even for small n), we resort to a more efficient method to sample plausible hypotheses using a sequential Monte Carlo (SMC) sampler. Although quite costly by common standards (a few minutes using a GPU in our experimental setup), we show in this work that SMC samplers remain tractable even for large n, opening new possibilities for group testing. In short, in return for a few minutes of computations, our detectives get an extensive list of thousands of relevant hypotheses that may explain tests observed so far.

Equipped with a relevant list of hypotheses, our strategy proceeds, as detectives would, by selectively gathering additional evidence. If k tests can be carried out at the next iteration, our strategy will propose to test k new groups, which are computed using the framework of Bayesian optimal experimental design. Intuitively, if k=1 and one can only propose a single new group to test, there would be clear advantage in building that group such that its test outcome is as uncertain as possible, i.e., with a probability that it returns positive as close to 50% as possible, given the current set of hypotheses. Indeed, to progress in an investigation, it is best to maximize the surprise factor (or information gain) provided by new test results, as opposed to using them to confirm further what we already hold to be very likely. To generalize that idea to a set of k>1 new groups, we score this surprise factor by computing the mutual information of these “virtual” group tests vs. the distribution of hypotheses. We also consider a more involved approach that computes the expected area under the ROC curve (AUC) one would obtain from testing these new groups using the distribution of hypotheses. The maximization of these two criteria is carried out using a greedy approach, resulting in two group selectors, GMIMAX and GAUCMAX (greedy maximization of mutual information or AUC, respectively).

wet_labsamplergroup selector

Our group testing framework describes an interaction between a testing environment, the wet_lab, whose pooled test results are used by the sampler to draw thousands of plausible hypotheses on the infection status of all individuals. These hypotheses are then used by an optimization procedure, group_selector, that figures out what groups may be the most relevant to test in order to narrow down on the true infection status. Once formed, these new groups are then tested again, closing the loop. At any point in the procedure, the hypotheses formed by the sampler can be averaged to obtain the average probability of infection for each patient. From these probabilities, a decision on whether a patient is infected or not can be done by thresholding these probabilities at a certain confidence level.

Benchmarking origami assaysinformative Dorfman approach

We executed 5000 simulations on a sample population of 70 individuals with an infection rate of 2%. We have assumed sensitivity/specificity values of 85% / 97% for tests with groups of maximal size 10, which are representative of current PCR machines. This figure demonstrates that our approach outperforms the other baselines with as few as 24 tests (up to 8 tests used in 3 cycles), including both adaptive and non-adaptive varieties, and performs significantly better than individual tests (plotted in the sensitivity/specificity plane as a hexagon, requiring 70 tests), highlighting the savings potential offered by group testing. See preprint for other setups.

Conclusion various institutionsAcknowledgements We would like to thank our collaborators on this work, Olivier Teboul, in particular, for his help preparing figures, as well as Arnaud Doucet and Quentin BerthetWe also thankKevin Murphy and Olivier Bousquet (Google) for their suggestions at the earliest stages of this project, as well as Dan Popovici for his unwavering support pushing this forward; Ignacio Anegon, Jeremie Poschmann and Laurent Tesson (INSERM) for providing us background information on RT-PCR tests and Nicolas Chopin (CREST) for giving guidance on his work to define SMCs for binary spaces.

Google at ICML 2020

Monday, July 13, 2020

Posted by Jaqui Herman and Cat Armato, Program ManagersPlatinum SponsorInternational Conference on Machine LearningboldedICML ExpoGoogle Dataset Search: Building an Open Ecosystem for Dataset DiscoveryNatasha NoyEnd-to-end Bayesian inference workflows in TensorFlow ProbabilityColin Carroll PublicationsPopulation-Based Black-Box Optimization for Biological Sequence DesignChristof Angermueller, David Belanger, Andreea Gane, Zelda Mariet, David Dohan, Kevin Murphy, Lucy Colwell, D SculleyPredictive Coding for Locally-Linear ControlRui Shu, Tung Nguyen, Yinlam Chow, Tuan Pham, Khoat Than, Mohammad Ghavamzadeh, Stefano Ermon, Hung BuiFedBoost: A Communication-Efficient Algorithm for Federated LearningJenny Hamer, Mehryar Mohri, Ananda Theertha SureshFaster Graph Embeddings via CoarseningMatthew Fahrbach, Gramoz Goranci, Richard Peng, Sushant Sachdeva, Chi WangRevisiting Fundamentals of Experience ReplayWilliam Fedus, Prajit Ramachandran, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, Will DabneyBoosting for Control of Dynamical SystemsNaman Agarwal, Nataly Brukhim, Elad Hazan, Zhou LuNeural Clustering ProcessesAri Pakman, Yueqi Wang, Catalin Mitelut, JinHyung Lee, Liam PaninskiThe Tree Ensemble Layer: Differentiability Meets Conditional ComputationHussein Hazimeh, Natalia Ponomareva, Petros Mol, Zhenyu Tan, Rahul MazumderRepresentations for Stable Off-Policy Reinforcement LearningDibya Ghosh, Marc BellemareREALM: Retrieval-Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei ChangContext Aware Local Differential PrivacyJayadev Acharya, Keith Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng SunScalable Deep Generative Modeling for Sparse GraphsHanjun Dai, Azade Nazi, Yujia Li, Bo Dai, Dale SchuurmansDeep k-NN for Noisy LabelsDara Bahri, Heinrich Jiang, Maya Gupta^†Revisiting Spatial Invariance with Low-Rank Local ConnectivityGamaleldin F. Elsayed, Prajit Ramachandran, Jonathon Shlens, Simon KornblithSCAFFOLD: Stochastic Controlled Averaging for Federated LearningSai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha SureshIncremental Sampling Without Replacement for Sequence ModelsKensen Shi, David Bieber, Charles SuttonSoftSort: A Continuous Relaxation for the argsort OperatorSebastian Prillo, Julian Martin EisenschlosXTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation(see blog post)Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin JohnsonLearning to Stop While Learning to PredictXinshi Chen, Hanjun Dai, Yu Li, Xin Gao, Le SongBandits with Adversarial ScalingThodoris Lykouris, Vahab Mirrokni, Renato Paes LemeSimGANs: Simulator-Based Generative Adversarial Networks for ECG Synthesis to Improve Deep ECG ClassificationTomer Golany, Daniel Freedman, Kira RadinskyStochastic Frank-Wolfe for Constrained Finite-Sum MinimizationGeoffrey Negiar, Gideon Dresdner, Alicia Yi-Ting Tsai, Laurent El Ghaoui, Francesco Locatello, Robert M. Freund, Fabian PedregosaImplicit differentiation of Lasso-type models for hyperparameter optimizationQuentin Bertrand, Quentin Klopfenstein, Mathieu Blondel, Samuel Vaiter, Alexandre Gramfort, Joseph SalmonInfinite attention: NNGP and NTK for deep attention networksJiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein, Roman NovakLogarithmic Regret for Learning Linear Quadratic Regulators EfficientlyAsaf Cassel, Alon Cohen, Tomer KorenAdversarial Learning Guarantees for Linear Hypotheses and Neural NetworksPranjal Awasthi, Natalie Frank, Mehryar MohriRandom Hypervolume Scalarizations for Provable Multi-Objective Black Box OptimizationDaniel Golovin, Qiuyi (Richard) ZhangGenerating Programmatic Referring Expressions via Program SynthesisJiani Huang, Calvin Smith, Osbert Bastani, Rishabh Singh, Aws Albarghouthi, Mayur NaikOptimizing Long-term Social Welfare in Recommender Systems: A Constrained Matching ApproachMartin Mladenov, Elliot Creager, Omer Ben-Porat, Kevin Swersky, Richard Zemel, Craig BoutilierAutoML-Zero: Evolving Machine Learning Algorithms From Scratch(see blog post)Esteban Real, Chen Liang, David R. So, Quoc V. LeHow Good is the Bayes Posterior in Deep Neural Networks Really?Florian Wenzel, Kevin Roth, Bastiaan S. Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, Sebastian Nowozin^†Which Tasks Should Be Learned Together in Multi-task Learning?Trevor Standley, Amir R. Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, Silvio SavareseInfluence Diagram Bandits: Variational Thompson Sampling for Structured Bandit ProblemsTong Yu, Branislav Kveton, Zheng Wen, Ruiyi Zhang, Ole J. MengshoelDisentangling Trainability and Generalization in Deep Neural NetworksLechao Xiao, Jeffrey Pennington, Samuel S. SchoenholzThe Many Shapley Values for Model ExplanationMukund Sundararajan, Amir NajmiNeural Contextual Bandits with UCB-based ExplorationDongruo Zhou, Lihong Li, Quanquan GuAutomatic Shortcut Removal for Self-Supervised Representation LearningMatthias Minderer, Olivier Bachem, Neil Houlsby, Michael TschannenFederated Learning with Only Positive LabelsFelix X. Yu, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv KumarHow Recurrent Networks Implement Contextual Processing in Sentiment AnalysisNiru Maheswaranathan, David SussilloSupervised Learning: No Loss No CryRichard Nock, Aditya Krishna MenonReady Policy One: World Building Through Active LearningPhilip Ball, Jack Parker-Holder, Aldo Pacchiano, Krzysztof Choromanski, Stephen RobertsWeakly-Supervised Disentanglement Without CompromisesFrancesco Locatello, Ben Poole, Gunnar Raetsch, Bernhard Schölkopf, Olivier Bachem, Michael TschannenFast Differentiable Sorting and RankingMathieu Blondel, Olivier Teboul, Quentin Berthet, Josip DjolongaDebiased Sinkhorn barycentersHicham Janati, Marco Cuturi, Alexandre GramfortInterpretable, Multidimensional, Multimodal Anomaly Detection with Negative Sampling for Detection of Device FailureJohn SippleAccelerating Large-Scale Inference with Anisotropic Vector QuantizationRuiqi Guo, Philip Sun, Erik Lindgren, Quan Geng^†, David Simcha, Felix Chern, Sanjiv KumarAn Optimistic Perspective on Offline Reinforcement Learning(see blog post)Rishabh Agarwal, Dale Schuurmans, Mohammad NorouziThe Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of GeneralizationBen Adlam, Jeffrey PenningtonPrivate Query Release Assisted by Public DataRaef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, Zhiwei Steven WuLearning and Evaluating Contextual Embedding of Source CodeAditya Kanade, Petros Maniatis, Gogul Balakrishnan, Kensen ShiEvaluating Machine Accuracy on ImageNetVaishaal Shankar, Rebecca Roelofs, Horia Mania, Alex Fang, Benjamin Recht, Ludwig SchmidtImputer: Sequence Modelling via Imputation and Dynamic ProgrammingWilliam Chan, Chitwan Saharia, Geoffrey Hinton, Mohammad Norouzi, Navdeep JaitlyDomain Aggregation Networks for Multi-Source Domain AdaptationJunfeng Wen, Russell Greiner, Dale SchuurmansPlanning to Explore via Self-Supervised World ModelsRamanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak PathakContext-Aware Dynamics Model for Generalization in Model-Based Reinforcement LearningKimin Lee, Younggyo Seo, Seunghyun Lee, Honglak Lee, Jinwoo ShinRetro*: Learning Retrosynthetic Planning with Neural Guided A* SearchBinghong Chen, Chengtao Li, Hanjun Dai, Le SongOn the Consistency of Top-k Surrogate LossesForest YangSanmi KoyejoDual Mirror Descent for Online Allocation ProblemsHaihao Lu, Santiago Balseiro, Vahab MirrokniEfficient and Scalable Bayesian Neural Nets with Rank-1 FactorsMichael W. Dusenberry, Ghassen Jerfel, Yeming Wen, Yi-An Ma^†, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, Dustin TranBatch Stationary Distribution EstimationJunfeng Wen, Bo Dai, Lihong Li, Dale SchuurmansSmall-GAN: Speeding Up GAN Training Using Core-SetsSamarth Sinha, Han Zhang, Anirudh Goyal, Yoshua Bengio, Hugo Larochelle, Augustus OdenaData Valuation Using Reinforcement LearningJinsung Yoon, Sercan ‎Ö. Arik, Tomas PfisterA Game Theoretic Perspective on Model-Based Reinforcement LearningAravind Rajeswaran, Igor Mordatch, Vikash KumarEncoding Musical Style with Transformer AutoencodersKristy Choi, Curtis Hawthorne, Ian Simon, Monica Dinculescu, Jesse EngelThe Shapley Taylor Interaction IndexKedar Dhamdhere, Mukund Sundararajan, Ashish AgarwalMultidimensional Shape ConstraintsMaya Gupta^†, Erez Louidor, Olexander Mangylov^†, Nobu Morioka, Taman Narayan, Sen ZhaoPrivate Counting from Anonymous Messages: Near-Optimal Accuracy with Vanishing Communication OverheadBadih Ghazi, Ravi Kumar, Pasin Manurangsi, Rasmus PaghLearning to Score Behaviors for Guided Policy OptimizationAldo Pacchiano, Jack Parker-Holder, Yunhao Tang, Anna Choromanska, Krzysztof Choromanski Michael I. JordanFundamental Tradeoffs between Invariance and Sensitivity to Adversarial PerturbationsFlorian Tramèr, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, Jörn-Henrik JacobsenOptimizing Black-Box Metrics with Adaptive SurrogatesQijia Jiang, Olaoluwa Adigun, Harikrishna Narasimhan, Mahdi Milani Fard, Maya Gupta^†Circuit-Based Intrinsic Methods to Detect OverfittingSat Chatterjee, Alan MishchenkoAutomatic Reparameterisation of Probabilistic ProgramsMaria I. Gorinova, Dave Moore, Matthew D. HoffmanStochastic Flows and Geometric Optimization on the Orthogonal GroupKrzysztof Choromanski, David Cheikhi, Jared Davis, Valerii Likhosherstov, Achille Nazaret, Achraf Bahamou, Xingyou Song, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas SindhwaniBlack-Box Variational Inference as a Parametric Approximation to Langevin DynamicsMatthew Hoffman, Yi-An Ma^†Concise Explanations of Neural Networks Using Adversarial TrainingPrasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Somesh Jha, Xi Wup-Norm Flow Diffusion for Local Graph ClusteringShenghao Yang, Di Wang, Kimon FountoulakisEmpirical Study of the Benefits of Overparameterization in Learning Latent Variable ModelsRares-Darius Buhai, Yoni Halpern, Yoon Kim, Andrej Risteski, David SontagRobust Pricing in Dynamic Mechanism DesignYuan Deng, Sébastien Lahaie, Vahab MirrokniDifferentiable Product Quantization for Learning Compact Embedding LayersTing Chen, Lala Li, Yizhou SunAdaptive Region-Based Active LearningCorinna Cortes, Giulia DeSalvo, Claudio Gentile, Mehryar Mohri, Ningshan ZhangCountering Language Drift with Seeded Iterated LearningYuchen Lu, Soumye Singhal, Florian Strub, Olivier Pietquin, Aaron CourvilleDoes Label Smoothing Mitigate Label Noise?Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv KumarAcceleration Through Spectral Density EstimationFabian Pedregosa, Damien ScieurMomentum Improves Normalized SGDAshok Cutkosky, Harsh MehtaConQUR: Mitigating Delusional Bias in Deep Q-LearningAndy Su, Jayden Ooi, Tyler Lu, Dale Schuurmans, Craig BoutilierOnline Learning with Imperfect Hints Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, Manish PurohitGo Wide, Then Narrow: Efficient Training of Deep Thin NetworksDenny Zhou, Mao Ye, Chen Chen, Tianjian Meng, Mingxing Tan, Xiaodan Song, Quoc Le, Qiang Liu, Dale SchuurmansOn Implicit Regularization in β-VAEsAbhishek Kumar, Ben PooleIs Local SGD Better than Minibatch SGD?Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan SrebA Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey HintonUniversal Average-Case Optimality of Polyak MomentumDamien Scieur, Fabian PedregosaAn Imitation Learning Approach for Cache ReplacementEvan Zheran Liu, Milad Hashemi, Kevin Swersky, Parthasarathy Ranganathan, Junwhan AhnCollapsed Amortized Variational Inference for Switching Nonlinear Dynamical SystemsZhe Dong, Bryan A. Seybold, Kevin P. Murphy, Hung H. BuiBeyond Synthetic Noise: Deep Learning on Controlled Noisy LabelsLu Jiang, Di Huang, Mason Liu, Weilong YangOptimizing Data Usage via Differentiable RewardsXinyi Wang, Hieu Pham, Paul Michel, Antonios Anastasopoulos, Jaime Carbonell, Graham NeubigSparse Sinkhorn AttentionYi Tay, Dara Bahri, Liu Yang, Donald Metzler, Da-Cheng JuanOne Policy to Control Them All: Shared Modular Policies for Agent-Agnostic ControlWenlong Huang, Igor Mordatch, Deepak PathakOn Thompson Sampling with Langevin AlgorithmsEric Mazumdar, Aldo Pacchiano, Yi-An Ma^†, Peter L. Bartlett, Michael I. JordanGood Subnetworks Provably Exist: Pruning via Greedy Forward SelectionMao Ye, Chengyue Gong, Lizhen Nie, Denny Zhou, Adam Klivans, Qiang LiuOn the Global Convergence Rates of Softmax Policy Gradient MethodsJincheng Mei, Chenjun Xiao, Csaba Szepesvari, Dale SchuurmansConcept Bottleneck ModelsPang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy LiangSupervised Quantile Normalization for Low-Rank Matrix ApproximationMarco Cuturi, Olivier Teboul, Jonathan Niles-Weed, Jean-Philippe VertMissing Data Imputation Using Optimal TransportBoris Muzellec, Julie Josse, Claire Boyer, Marco CuturiLearning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention Over ModulesSarthak Mittal, Alex Lamb, Anirudh Goyal, Vikram Voleti, Murray Shanahan, Guillaume Lajoie, Michael Mozer, Yoshua BengioStochastic Optimization for Regularized Wasserstein EstimatorsMarin Ballu, Quentin Berthet, Francis BachLow-Rank Bottleneck in Multi-head Attention ModelsSrinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank Jakkam Reddi, Sanjiv KumarRigging the Lottery: Making All Tickets WinnersUtku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, Erich ElsenOnline Learning with Dependent Stochastic Feedback GraphsCorinna Cortes, Giulia DeSalvo, Claudio Gentile, Mehryar Mohri, Ningshan ZhangCalibration, Entropy Rates, and Memory in Language ModelsMark Braverman, Xinyi Chen, Sham Kakade, Karthik Narasimhan, Cyril Zhang, Yi Zhang Composable Sketches for Functions of Frequencies: Beyond the Worst CaseEdith Cohen, Ofir Geri, Rasmus PaghEnergy-Based Processes for Exchangeable DataMengjiao Yang, Bo Dai, Hanjun Dai, Dale SchuurmansNear-Optimal Regret Bounds for Stochastic Shortest PathAlon Cohen, Haim Kaplan, Yishay Mansour, Aviv RosenbergPEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization(see blog post)Jingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. LiuThe Complexity of Finding Stationary Points with Stochastic Gradient DescentYoel Drori, Ohad ShamirThe k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural NetworksJakub Swiatkowski, Kevin Roth, Bas Veeling, Linh Tran, Josh Dillon, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, Sebastian Nowozin^†Regularized Optimal Transport is Ground Cost AdversarialFrançois-Pierre Paty, Marco CuturiWorkshopsNew In ML Nicolas Le RouxZhen Xu, Sparkle Russell-Puleri, Zhengying Liu, Sinead A Williamson, Matthias W Seeger, Wei-Wei Tu, Samy Bengio, Isabelle GuyonLatinX in AI Pablo Samuel CastroWomen in Machine Learning Un-Workshop Doina Precup Jennifer WeiQueer in AI Shakir MohamedWorkshop on Continual Learning Haytham Fayek, Arslan Chaudhry, David Lopez-Paz, Eugene Belilovsky, Jonathan Schwarz, Marc Pickett, Rahaf Aljundi, Sayna Ebrahimi, Razvan Pascanu, Puneet Dokania5th ICML Workshop on Human Interpretability in Machine Learning (WHI) Kush Varshney, Adrian Weller, Alice Xiang, Amit Dhurandhar, Been Kim, Dennis Wei, Umang BhattSelf-supervision in Audio and Speech Mirco Ravanelli, Dmitriy Serdyuk, R Devon Hjelm, Bhuvana Ramabhadran, Titouan ParcolletWorkshop on eXtreme Classification: Theory and Applications Sanjiv KumarHealthcare Systems, Population Health, and the Role of Health-tech Krzysztof Choromanski, David Cheikhi, Jared Davis, Valerii Likhosherstov, Achille Nazaret, Achraf Bahamou, Xingyou Song, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas SindhwaniTheoretical Foundations of Reinforcement Learning Alon Cohen, Chris DannUncertainty and Robustness in Deep Learning Workshop (UDL) Justin GilmerSharon Li, Balaji Lakshminarayanan, Dan Hendrycks, Thomas Dietterich, Jasper Snoek Jeremiah Liu, Jie Ren, Rodolphe Jenatton, Zack Nado, Alexander Alemi, Florian Wenzel, Mike Dusenberry, Raphael Lopes Beyond First Order Methods in Machine Learning Systems Jonathan HseuObject-Oriented Learning: Perception, Representation, and Reasoning Thomas Kipf, Igor Mordatch Graph Representation Learning and Beyond (GRL+) Michael Bronstein, Andreea Deac, William L. Hamilton, Jessica B. Hamrick, Milad Hashemi, Stefanie Jegelka, Jure Leskovec, Renjie Liao, Federico Monti, Yizhou Sun, Kevin Swersky, Petar Veličković, Rex Ying, Marinka Žitnik Thomas Kipf Bryan Perozzi, Kevin Swersky, Milad Hashemi, Thomas Kipf, Ting ChengML Interpretability for Scientific Discovery Subhashini Venugopalan, Michael Brenner, Scott Linderman, Been Kim Akinori Mitani, Arunachalam Narayanaswamy, Avinash Varadarajan, Awa Dieng, Benjamin Sanchez-Lengeling, Bo Dai, Stephan Hoyer, Subham Sekhar Sahoo, Suhani Vora John Platt, Mukund Sundararajan, Jon KleinbergNegative Dependence and Submodularity for Machine Learning Zelda Mariet, Mike Gartrell, Michal Derezinski7th ICML Workshop on Automated Machine Learning (AutoML) Charles Weill, Katharina Eggensperger, Matthias Feurer, Frank Hutter, Marius Lindauer, Joaquin VanschorenFederated Learning for User Privacy and Data Confidentiality Brendan McMahan Peter Kairouz, Jakub Konecný MLRetrospectives: A Venue for Self-Reflection in ML Research Margaret MitchellMachine Learning for Media DiscoveryEd ChiINNF+: Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models: Chin-Wei Huang, David Krueger, Rianne van den Berg, George Papamakarios, Chris Cremer, Ricky Chen, Danilo Rezende4th Lifelong Learning Workshop George Tucker, Marlos C. Machado2nd ICML Workshop on Human in the Loop Learning (HILL)Organizers: Shanghang Zhang, Xin Wang, Fisher Yu, Jiajun Wu, Trevor DarrellMachine Learning for Global Health: Danielle Belgrave, Danielle Belgrave, Stephanie Hyland, Charles Onu, Nicholas Furnham, Ernest Mwebaze, Neil LawrenceCommittee Adam White^†Work performed while at Google

Grounding Natural Language Instructions to Mobile UI Actions

Friday, July 10, 2020

Posted by Yang Li, Research Scientist, Google ResearchMapping Natural Language Instructions to Mobile UI Action SequencesACL 2020automatic action sequence mappingGitHub repositoryConstructing Language Grounding Modelslanguage groundingaction phrase-extractiongrounding

The workflow of grounding language instructions to executable actions.

Transformerarea attention

The action phrase extraction model takes a word sequence of a natural language instruction and outputs a sequence of spans (denoted in red boxes) that indicate the phrases describing the operation, the object and the argument of each action in the task.

The grounding model takes the extracted spans as input and grounds them to executable actions, including the object an action is applied to, given the UI screen at each step during execution.

Resultspublic android UI corpusgraph convolutional networkfeedforward networkConclusionAcknowledgements Many thanks to my co-authors Jiacong He, Xin Zhou, Yuan Zhang and Jason Baldridge on this work at Google Research. I would also like to thank Gang Li who provided generous help for creating open-source datasets, and Ashwin Kakarla, Muqthar Mohammad and Mohd Majeed for their help with the annotations.

AutoML-Zero: Evolving Code that Learns

Thursday, July 9, 2020

Posted by Esteban Real, Staff Software Engineer, and Chen Liang, Software Engineer, Google Research, Brain Teamdeep neural networksAutoMLneural architecture searchtopicmuchresearchlearning rulesince the early 90sour researchrecent paperAutoML-Zerobackpropagationlinear regressionEvolving Learning Algorithms from Scratchvariantevolutionary methodsproved usefulsince the 80smakes them especially suitable for the discovery of learning algorithmsparentmutated

A population is initialized with empty programs. Many generations later, we see a more evolved population and two of its algorithms compete. The most accurate wins to produce a child. After many such events, the final population contains highly accurate classifiers.

Exploring a Difficult Search Space12 migrationcodestochastic gradient descent

Progress of an evolution experiment. As time passes, from left to right, we see the algorithms becoming more complex and more accurate.

The Evolved Algorithmnoise injection as data augmentationbilinear modelgradient normalizationweight averagingnoisy ReLUConclusion^*AcknowledgementsWe want to thank our co-authors, David R. So and Quoc V. Le, and the many who helped us through discussions during the project and paper writing, including Samy Bengio, Vincent Vanhoucke, Doug Eck, Charles Sutton, Yanping Huang, Jacques Pienaar, Jeff Dean, and particularly Gabriel Bender, Hanxiao Liu, Rishabh Singh, Chiyuan Zhang, and Hieu Pham. We also want to especially thank Tom Small for contributing the animations in this post.* The electricity consumption for the experiments (run in 2019) was matched with the purchase of renewable energy. ↩

Blog