Groups across Google are actively pursuing research across the field of machine learning, ranging from theory to application. With scalable tools and architectures, we build machine learning systems to solve deep scientific and engineering challenges in areas of language, music, visual processing, and more.
Google is proud to be a Platinum Sponsor of the thirty-eighth International Conference on Machine Learning (ICML 2021), a premier annual event happening this week. As a leader in machine learning research — with over 100 accepted publications and Googlers participating in workshops — we look forward to our continued partnership with the broader machine learning research community.
Registered for ICML 2021? We hope you’ll visit the Google virtual booth to learn more about the exciting work, creativity, and fun that goes into solving a portion of the field’s most interesting challenges. Take a look below to learn more about the Google research being presented at ICML 2021 (Google affiliations in bold).
Organizing Committee ICML Board Members include: Corinna Cortes, Hugo Larochelle, Shakir Mohamed ICML Emeritus Board includes: William Cohen, Andrew McCallum Tutorial Co-Chair member: Quoc Lee
Publications Attention Is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth Yihe Dong, Jean-Baptiste Cordonnier, Andreas Loukas
Scalable Evaluation of Multi-agent Reinforcement Learning with Melting Pot Joel Z. Leibo, Edgar Duéñez-Guzmán, Alexander Sasha Vezhnevets, John P. Agapiou, Peter Sunehag, Raphael Koster, Jayd Matyas, Charles Beattie, Igor Mordatch, Thore Graepel
On the Optimality of Batch Policy Optimization Algorithms Chenjun Xiao, Yifan Wu, Tor Lattimore, Bo Dai, Jincheng Mei, Lihong Li*, Csaba Szepesvari, Dale Schuurmans
Low-Rank Sinkhorn Factorization Meyer Scetbon, Marco Cuturi, Gabriel Peyré
Oops I Took A Gradient: Scalable Sampling for Discrete Distributions Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, Chris J. Maddison
PID Accelerated Value Iteration Algorithm Amir-Massoud Farahmand, Mohammad Ghavamzadeh
Dueling Convex Optimization Aadirupa Saha, Tomer Koren, Yishay Mansour
What Are Bayesian Neural Network Posteriors Really Like? Pavel Izmailov, Sharad Vikram, Matthew D. Hoffman, Andrew Gordon Wilson
Offline Reinforcement Learning with Pseudometric Learning Robert Dadashi, Shideh Rezaeifar, Nino Vieillard, Léonard Hussenot, Olivier Pietquin, Matthieu Geist
Revisiting Rainbow: Promoting More Insightful and Inclusive Deep Reinforcement Learning Research (see blog post) Johan S. Obando-Ceron, Pablo Samuel Castro
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL Seyed Kamyar Seyed Ghasemipour*, Dale Schuurmans, Shixiang Shane Gu
Variational Data Assimilation with a Learned Inverse Observation Operator Thomas Frerix, Dmitrii Kochkov, Jamie A. Smith, Daniel Cremers, Michael P. Brenner, Stephan Hoyer
Tilting the Playing Field: Dynamical Loss Functions for Machine Learning Miguel Ruiz-Garcia, Ge Zhang, Samuel S. Schoenholz, Andrea J. Liu
Model-Based Reinforcement Learning via Latent-Space Collocation Oleh Rybkin, Chuning Zhu, Anusha Nagabandi, Kostas Daniilidis, Igor Mordatch, Sergey Levine
Momentum Residual Neural Networks Michael E. Sander, Pierre Ablin, Mathieu Blondel, Gabriel Peyré
OmniNet: Omnidirectional Representations from Transformers Yi Tay, Mostafa Dehghani, Vamsi Aribandi, Jai Gupta, Philip Pham, Zhen Qin, Dara Bahri, Da-Cheng Juan, Donald Metzler
Synthesizer: Rethinking Self-Attention for Transformer Models Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, Che Zheng
Towards Domain-Agnostic Contrastive Learning Vikas Verma, Minh-Thang Luong, Kenji Kawaguchi, Hieu Pham, Quoc V. Le
Randomized Entity-wise Factorization for Multi-agent Reinforcement Learning Shariq Iqbal, Christian A. Schroeder de Witt, Bei Peng, Wendelin Böhmer, Shimon Whiteson, Fei Sha
LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning Yuhuai Wu, Markus Rabe, Wenda Li, Jimmy Ba, Roger Grosse, Christian Szegedy
Emergent Social Learning via Multi-agent Reinforcement Learning Kamal Ndousse, Douglas Eck, Sergey Levine, Natasha Jaques
Improved Contrastive Divergence Training of Energy-Based Models Yilun Du, Shuang Li, Joshua Tenenbaum, Igor Mordatch
Characterizing Structural Regularities of Labeled Data in Overparameterized Models Ziheng Jiang*, Chiyuan Zhang, Kunal Talwar, Michael Mozer
Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, Sergey Levine
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning Angelos Filos, Clare Lyle, Yarin Gal, Sergey Levine, Natasha Jaques, Gregory Farquhar
EfficientNetV2: Smaller Models and Faster Training (see blog post) Mingxing Tan, Quoc V. Le
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies Paul Vicol, Luke Metz, Jascha Sohl-Dickstein
Federated Composite Optimization Honglin Yuan*, Manzil Zaheer, Sashank Reddi
Light RUMs Flavio Chierichetti, Ravi Kumar, Andrew Tomkins
Catformer: Designing Stable Transformers via Sensitivity Analysis Jared Quincy Davis, Albert Gu, Krzysztof Choromanski, Tri Dao, Christopher Re, Chelsea Finn, Percy Liang
Representation Matters: Offline Pretraining for Sequential Decision Making Mengjiao Yang, Ofir Nachum
Variational Empowerment as Representation Learning for Goal-Conditioned Reinforcement Learning Jongwook Choi*, Archit Sharma*, Honglak Lee, Sergey Levine, Shixiang Shane Gu
Beyond Variance Reduction: Understanding the True Impact of Baselines on Policy Optimization Wesley Chung, Valentin Thomas, Marlos C. Machado, Nicolas Le Roux
Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization Neha S. Wadia, Daniel Duckworth, Samuel S. Schoenholz, Ethan Dyer, Jascha Sohl-Dickstein
Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers Piotr Teterwak*, Chiyuan Zhang, Dilip Krishnan, Michael C. Mozer
Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning Hiroki Furuta, Tatsuya Matsushima, Tadashi Kozuno, Yutaka Matsuo, Sergey Levine, Ofir Nachum, Shixiang Shane Gu
Hyperparameter Selection for Imitation Learning Leonard Hussenot, Marcin Andrychowicz, Damien Vincent, Robert Dadashi, Anton Raichuk, Lukasz Stafiniak, Sertan Girgin, Raphael Marinier, Nikola Momchev, Sabela Ramos, Manu Orsini, Olivier Bachem, Matthieu Geist, Olivier Pietquin
Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum, Sadeep Jayasumana, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing Yuan Deng, Sebastien Lahaie, Vahab Mirrokni, Song Zuo
Debiasing a First-Order Heuristic for Approximate Bi-Level Optimization Valerii Likhosherstov, Xingyou Song, Krzysztof Choromanski, Jared Davis, Adrian Weller
Characterizing the Gap Between Actor-Critic and Policy Gradient Junfeng Wen, Saurabh Kumar, Ramki Gummadi, Dale Schuurmans
Composing Normalizing Flows for Inverse Problems Jay Whang, Erik Lindgren, Alexandros Dimakis
Online Policy Gradient for Model Free Learning of Linear Quadratic Regulators with √T Regret Asaf Cassel, Tomer Koren
Learning to Price Against a Moving Target Renato Paes Leme, Balasubramanian Sivan, Yifeng Teng, Pratik Worah
Fairness and Bias in Online Selection Jose Correa, Andres Cristi, Paul Duetting, Ashkan Norouzi-Fard
The Impact of Record Linkage on Learning from Feature Partitioned Data Richard Nock, Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Jakub Nabaglo, Giorgio Patrini, Guillaume Smith, Brian Thorne
Reserve Price Optimization for First Price Auctions in Display Advertising Zhe Feng*, Sébastien Lahaie, Jon Schneider, Jinchao Ye
A Regret Minimization Approach to Iterative Learning Control Naman Agarwal, Elad Hazan, Anirudha Majumdar, Karan Singh
A Statistical Perspective on Distillation Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar
Best Model Identification: A Rested Bandit Formulation Leonardo Cella, Massimiliano Pontil, Claudio Gentile
Generalised Lipschitz Regularisation Equals Distributional Robustness Zac Cranko, Zhan Shi, Xinhua Zhang, Richard Nock, Simon Kornblith
Stochastic Multi-armed Bandits with Unrestricted Delay Distributions Tal Lancewicki, Shahar Segal, Tomer Koren, Yishay Mansour
Regularized Online Allocation Problems: Fairness and Beyond Santiago Balseiro, Haihao Lu, Vahab Mirrokni
Implicit Rate-Constrained Optimization of Non-decomposable Objectives Abhishek Kumar, Harikrishna Narasimhan, Andrew Cotter
Leveraging Non-uniformity in First-Order Non-Convex Optimization Jincheng Mei, Yue Gao, Bo Dai, Csaba Szepesvari, Dale Schuurmans
Dynamic Balancing for Model Selection in Bandits and RL Ashok Cutkosky, Christoph Dann, Abhimanyu Das, Claudio Gentile, Aldo Pacchiano, Manish Purohit
Adversarial Dueling Bandits Aadirupa Saha, Tomer Koren, Yishay Mansour
Optimizing Black-Box Metrics with Iterative Example Weighting Gaurush Hiranandani*, Jatin Mathur, Harikrishna Narasimhan, Mahdi Milani Fard, Oluwasanmi Koyejo
Relative Deviation Margin Bounds Corinna Cortes, Mehryar Mohri, Ananda Theertha Suresh
MC-LSTM: Mass-Conserving LSTM Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitner, Grey Nearing, Sepp Hochreiter, Günter Klambauer
12-Lead ECG Reconstruction via Koopman Operators Authors:Tomer Golany, Kira Radinsky, Daniel Freedman, Saar Minha
Finding Relevant Information via a Discrete Fourier Expansion Mohsen Heidari, Jithin Sreedharan, Gil Shamir, Wojciech Szpankowski
LEGO: Latent Execution-Guided Reasoning for Multi-hop Question Answering on Knowledge Graphs Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Michihiro Yasunaga, Haitian Sun, Dale Schuurmans, Jure Leskovec, Denny Zhou
SpreadsheetCoder: Formula Prediction from Semi-structured Context Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, Denny Zhou
Combinatorial Blocking Bandits with Stochastic Delays Alexia Atsidakou, Orestis Papadigenopoulos, Soumya Basu, Constantine Caramani, Sanjay Shakkottai
Beyond log2(T) Regret for Decentralized Bandits in Matching Markets Soumya Basu, Karthik Abinav Sankararaman, Abishek Sankararaman
Robust Pure Exploration in Linear Bandits with Limited Budget Ayya Alieva, Ashok Cutkosky, Abhimanyu Das
Latent Programmer: Discrete Latent Codes for Program Synthesis Joey Hong, David Dohan, Rishabh Singh, Charles Sutton, Manzil Zaheer
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (see blog post) Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, Tom Duerig
On Linear Identifiability of Learned Representations Geoffrey Roeder, Luke Metz, Diederik P. Kingma
Hierarchical Clustering of Data Streams: Scalable Algorithms and Approximation Guarantees Anand Rajagopalan, Fabio Vitale, Danny Vainstein, Gui Citovsky, Cecilia M Procopiuc, Claudio Gentile
Differentially Private Quantiles Jennifer Gillenwater, Matthew Joseph, Alex Kulesza
Active Covering Heinrich Jiang, Afshin Rostamizadeh
Sharf: Shape-Conditioned Radiance Fields from a Single View Konstantinos Rematas, Ricardo Martin-Brualla, Vittorio Ferrari
Learning a Universal Template for Few-Shot Dataset Generalization Eleni Triantafillou*, Hugo Larochelle, Richard Zemel, Vincent Dumoulin
Private Alternating Least Squares: Practical Private Matrix Completion with Tighter Rates Steve Chien, Prateek Jain, Walid Krichene, Steffen Rendle, Shuang Song, Abhradeep Thakurta, Li Zhang
Differentially-Private Clustering of Easy Instances Edith Cohen, Haim Kaplan, Yishay Mansour, Uri Stemmer, Eliad Tsfadia
Label-Only Membership Inference Attacks Christopher A. Choquette-Choo, Florian Tramèr, Nicholas Carlini, Nicolas Papernot
Neural Feature Matching in Implicit 3D Representations Yunlu Chen, Basura Fernando, Hakan Bilen, Thomas Mensink, Efstratios Gavves
Locally Private k-Means in One Round Alisa Chang, Badih Ghazi, Ravi Kumar, Pasin Manurangsi
Large-Scale Meta-learning with Continual Trajectory Shifting Jaewoong Shin, Hae Beom Lee, Boqing Gong, Sung Ju Hwang
Statistical Estimation from Dependent Data Vardis Kandiros, Yuval Dagan, Nishanth Dikkala, Surbhi Goel, Constantinos Daskalakis
Oneshot Differentially Private Top-k Selection Gang Qiao, Weijie J. Su, Li Zhang
Unsupervised Part Representation by Flow Capsules Sara Sabour, Andrea Tagliasacchi, Soroosh Yazdani, Geoffrey E. Hinton, David J. Fleet
Private Stochastic Convex Optimization: Optimal Rates in L1 Geometry Hilal Asi, Vitaly Feldman, Tomer Koren, Kunal Talwar
Practical and Private (Deep) Learning Without Sampling or Shuffling Peter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, Zheng Xu
Differentially Private Aggregation in the Shuffle Model: Almost Central Accuracy in Almost a Single Message Badih Ghazi, Ravi Kumar, Pasin Manurangsi, Rasmus Pagh, Amer Sinha
Leveraging Public Data for Practical Private Query Release Terrance Liu, Giuseppe Vietri, Thomas Steinke, Jonathan Ullman, Zhiwei Steven Wu
Meta-Thompson Sampling Branislav Kveton, Mikhail Konobeev, Manzil Zaheer, Chih-wei Hsu, Martin Mladenov, Craig Boutilier, Csaba Szepesvári
Implicit-PDF: Non-parametric Representation of Probability Distributions on the Rotation Manifold Kieran A Murphy, Carlos Esteves, Varun Jampani, Srikumar Ramalingam, Ameesh Makadia
Improving Ultrametrics Embeddings Through Coresets Vincent Cohen-Addad, Rémi de Joannis de Verclos, Guillaume Lagarde
A Discriminative Technique for Multiple-Source Adaptation Corinna Cortes, Mehryar Mohri, Ananda Theertha Suresh, Ningshan Zhang
Self-Supervised and Supervised Joint Training for Resource-Rich Machine Translation Yong Cheng, Wei Wang*, Lu Jiang, Wolfgang Macherey
Correlation Clustering in Constant Many Parallel Rounds Vincent Cohen-Addad, Silvio Lattanzi, Slobodan Mitrović, Ashkan Norouzi-Fard, Nikos Parotsidis, Jakub Tarnawski
Hierarchical Agglomerative Graph Clustering in Nearly-Linear Time Laxman Dhulipala, David Eisenstat, Jakub Łącki, Vahab Mirrokni, Jessica Shi
Meta-learning Bidirectional Update Rules Mark Sandler, Max Vladymyrov, Andrey Zhmoginov, Nolan Miller, Andrew Jackson, Tom Madams, Blaise Aguera y Arcas
Discretization Drift in Two-Player Games Mihaela Rosca, Yan Wu, Benoit Dherin, David G.T. Barrett
Reasoning Over Virtual Knowledge Bases With Open Predicate Relations Haitian Sun*, Pat Verga, Bhuwan Dhingra, Ruslan Salakhutdinov, William W. Cohen
Learn2Hop: Learned Optimization on Rough Landscapes Amil Merchant, Luke Metz, Samuel Schoenholz, Ekin Cubuk
Locally Adaptive Label Smoothing Improves Predictive Churn Dara Bahri, Heinrich Jiang
Overcoming Catastrophic Forgetting by Bayesian Generative Regularization Patrick H. Chen, Wei Wei, Cho-jui Hsieh, Bo Dai
Workshops (only Google affiliations are noted) LatinX in AI (LXAI) Research at ICML 2021 Hosts: Been Kim, Natasha Jaques
Uncertainty and Robustness in Deep Learning Organizers: Balaji Lakshminarayanan, Jasper Snoek Invited Speaker: Dustin Tran
Reinforcement Learning for Real Life Organizers: Minmin Chen, Lihong Li Invited Speaker: Ed Chi
Interpretable Machine Learning in Healthcare Organizers: Alan Karthikesalingam Invited Speakers: Abhijit Guha Roy, Jim Winkens
The Neglected Assumptions in Causal Inference Organizer: Alexander D'Amour
ICML Workshop on Algorithmic Recourse Invited Speakers: Been Kim, Berk Ustun
A Blessing in Disguise: The Prospects and Perils of Adversarial Machine Learning Invited Speaker: Nicholas Carlini
Overparameterization: Pitfalls and Opportunities Organizers: Yasaman Bahri, Hanie Sedghi
Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine Learning (ITR3) Invited Speaker: Thomas Steinke
Beyond First-Order Methods in Machine Learning Systems Invited Speaker: Courtney Paquette
ICML 2021 Workshop: Self-Supervised Learning for Reasoning and Perception Invited Speaker: Chelsea Finn
Workshop on Reinforcement Learning Theory Invited Speaker: Bo Dai
Tutorials (only Google affiliations are noted) Responsible AI in Industry: Practical Challenges and Lessons Learned Organizers: Ben Packer
Online and Non-stochastic Control Organizers: Elad Hazan
Random Matrix Theory and ML (RMT +ML) Organizers: Fabian Pedregosa, Jeffrey Pennington, Courntey Paquette Self-Attention for Computer Vision Organizers: Prajit Ramachandran, Ashish Vaswani
* Indicates work done while at Google
