Google AI Blog: March 2020

Exploring New Ways to Support Faculty Research

Thursday, March 26, 2020

Posted by Maggie Johnson, VP, Google ResearchGoogle Faculty Research Award ProgramResearch Scholar ProgramAward for Inclusion Research ProgramLatin America Research AwardsPhD Fellowship ProgramVisiting Researcher Programthis pagepublication database

A Neural Weather Model for Eight-Hour Precipitation Forecasting

Wednesday, March 25, 2020

Posted by Nal Kalchbrenner and Casper Sønderby, Research Scientists, Google Research, Amsterdamdeep neural networksTPUsprevious researchnowcastingMetNet: A Neural Weather Model for Precipitation ForecastingNOAA

MetNet model predictions compared to the ground truth as measured by the NOAA multi-radar/multi-sensor system (MRMS). The MetNet model (top) displays the probabilities for 1 mm/hr precipitation predicted from 2 minutes to 480 minutes ahead, whereas the MRMS data (bottom) shows the areas receiving at least 1 mm/hr of precipitation over that same time period.

Neural Weather Modelmulti-radar/multi-sensor systemGeostationary Operational Environmental Satellite

Size of the input patch containing satellite and radar images (large, 1024 x 1024 km square) and of the output predicted radar image (small, 64 x 64 km square).

convolutional LSTMaxial self-attention

The architecture of the neural weather model, MetNet. The input satellite and radar images first pass through a spatial downsampler to reduce memory consumption. They are then processed by a convolutional LSTM at 15 minute intervals over the 90 minutes of input data. Then axial attention layers are used to make the network see the entirety of the input images.

Resultsprecipitation rate forecasting benchmarkHigh Resolution Rapid Refreshbaseline modeloptical flowTPUsF1-score

Performance evaluated in terms of F1-score at 1.0 mm/h precipitation rate (higher is better). The neural weather model (MetNet) outperforms the physics-based model (HRRR) currently operational in the US for timescales up to 8 hours ahead.

single

Comparison between the output from MetNet (top) and HRRR (bottom) to ground-truth (middle) as retrieved from the NOAA MRMS system. Notice that while the HRRR model predicts structure that appears to be more similar to that of the ground-truth, the structure predicted may be grossly incorrect.

HRRR (left) predicts a single potential future outcome (in red) out of the many possible outcomes, whereas MetNet (right) directly accounts for uncertainty by assigning probabilities over the future outcomes.

Future DirectionsAcknowledgementsThis project was done in collaboration with Lasse Espeholt, Jonathan Heek, Mostafa Dehghani, Avital Oliver, Tim Salimans, Shreya Agrawal and Jason Hickey. We would also like to thank Manoj Kumar, Wendy Shang, Dick Weissenborn, Cenk Gazen, John Burge, Stephen Hoyer, Lak Lakshmanan, Rob Carver, Carla Bromberg and Aaron Bell for useful discussions and Tom Small for help with the visualizations.

Massively Scaling Reinforcement Learning with SEED RL

Monday, March 23, 2020

Posted by Lasse Espeholt, Research Engineer, Google Research, AmsterdamGoDota 2agentsSEED RL: Scalable and Efficient Deep-RL with Accelerated Central InferenceacceleratorsGPUsTPUsGoogle Research FootballArcade Learning EnvironmentDeepMind LabGithubGPUsCurrent Distributed ArchitecturesIMPALAactorslearners

Example architecture for an earlier generation RL agent, IMPALA. Inference is done on the actors, usually using inefficient CPUs. Updated model parameters are frequently sent from the learner to the actors increasing bandwidth requirements.

Using CPUs for neural network inference is much less efficient and slower than using accelerators and becomes problematic as models become larger and more computationally expensive.

The bandwidth required for sending parameters and intermediate model states between the actors and learner can be a bottleneck.

Handling two completely different tasks on one machine (i.e., environment rendering and inference) is unlikely to utilize machine resources optimally.

SEED RL Architecturenetwork librarygRPCTensorFlow 2TPUs

Overview of the architecture of SEED RL. In contrast to the IMPALA architecture, the actors only take actions in environments. Inference is executed centrally by the learner on accelerators using batches of data from multiple actors.

V-tracepolicy gradientoff-policyon-policyR2D2Q-learningExperiments

Frames per second comparing IMPALA and various configurations of SEED RL on DeepMind Lab. SEED RL achieves 2.4M frames per second using 4,160 CPUs. Assuming the same speed, IMPALA would need 14,000 CPUs.

Episode return (i.e., the sum of rewards) over time on the DeepMind Lab game “explore_goal_locations_small” using IMPALA and SEED RL. With SEED RL, the time to train is significantly reduced.

The score of different architectures on the Google Research Football “Hard” task. We show that by using an input resolution and a larger model, the score is improved, and with more training, the model can significantly outperform the builtin AI.

paperAcknowledgements This project was done in collaboration with Raphaël Marinier, Piotr Stanczyk, Ke Wang, Marcin Andrychowicz and Marcin Michalski. We would also like to thank Tom Small for the visualizations.

Visual Transfer Learning for Robotic Manipulation

Friday, March 20, 2020

Posted by Yen-Chen Lin, Research Intern and Andy Zeng, Research Scientist, Robotics at Googleaffordancesaffordance-based manipulationgraspingpushingthrowing

Overview of affordance-based manipulation.

data collectiontrial and errorrecentdiscoveriestransfer learningnow availableLearning to See before Learning to Act: Visual Pre-training for Manipulationresearchers from MITICRA 2020exploration processgrippers

Does first learning to see, improve the speed at which a robot can learn to act? In this project, we study ways in which we can transfer knowledge learned from computer vision tasks (left) to robot manipulation tasks (right).

Transfer Learning for Affordance-Based Manipulationlabelsvalue of actionsdeep modelResNetImageNetCOCO

Affordance-based grasping models trained from scratch can struggle to pick up new objects after 60 minutes of training (left). With pre-training from visual tasks, our affordance-based grasping models can easily generalize to picking up new objects with less than 10 minutes of training, even when evaluated with different hardware (middle: suction, right: gripper).

Transfer Learning Can Improve Exploration

Number of successful grasps out of the first 50 attempts using: a random initialization of weights, backbone and head pre-trained on ImageNet, COCO pre-trained backbone only, and backbone and head trained on COCO.

Affordances predicted by different models from images of cluttered objects (a). (b) Random refers to a randomly initialized model. (c) ImageNet is a model with backbone pre-trained on ImageNet and a randomly initialized head. (d) Normal refers to a model pre-trained to detect pixels with surface normals close to the anti-gravity axis. (e) COCO is the modified segmentation model (MaskRCNN) trained on the COCO dataset. (f) Suction is a converged model learned from robot-environment interactions using the suction gripper.

Limitations and Future WorkAcknowledgementsThis research was done by Yen-Chen Lin (Ph.D. student at MIT), Andy Zeng, Shuran Song, Phillip Isola (faculty at MIT), and Tsung-Yi Lin, with special thanks to Johnny Lee and Ivan Krasin for valuable managerial support, Chad Richards for helpful feedback on writing, and Jonathan Thompson for fruitful technical discussions.

Introducing Dreamer: Scalable Reinforcement Learning Using World Models

Wednesday, March 18, 2020

Posted by Danijar Hafner, Student Researcher, Google Researchartificial agentsreinforcement learningModel-freeDQNAlphaStarreal-world scenariosmodel-basedworld modelDeep Planning NetworkDeepMindDreamerbackpropagationcompact model statessource codeHow Does Dreamer Work?typical

The three processes of the Dreamer agent. The world model is learned from past experience. From predictions of this model, the agent then learns a value network to predict future rewards and an actor network to select actions. The actor network is used to interact with the environment.

Learning the World ModelPlaNetcompact model states

Dreamer learns a world model from experience. Using past images (o₁–o₃) and actions (a₁–a₂), it computes a sequence of compact model states (green circles) from which it reconstructs the images (ô₁–ô₃) and predicts the rewards (r̂₁–r̂₃).

DeepMind Control SuiteDeepMind Lab

Predicting ahead using compact model states enables long-term predictions in complex environments. Shown here are two sequences that the agent has not encountered before. Given five input images, the model reconstructs them and predicts the future images up to time step 50.

Efficient Behavior Learning

Dreamer learns long-sighted behaviors from predicted sequences of model states. It first learns the long-term value (v̂₂–v̂₃) of each state, and then predicts actions (â₁–â₂) that lead to high rewards and values by backpropagating them through the state sequence to the actor network.

Performance on Control Tasks

Dreamer learns to solve 20 challenging continuous control tasks with image inputs, 5 of which are displayed here. The visualizations show the same 64x64 images that the agent receives from the environment.

PlaNetA3CD4PG

Dreamer outperforms the previous best model-free (D4PG) and model-based (PlaNet) methods on the benchmark of 20 tasks in terms of final performance, data efficiency, and computation time.

Dreamer learns successful behaviors on Atari games and DeepMind Lab levels, which feature discrete actions and visually more diverse scenes, including 3D environments with multiple objects.

ConclusionAcknowledgementsThis project is a collaboration with Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. We further thank everybody in the Brain Team and beyond who commented on our paper draft and provided feedback at any point throughout the project.

Fast and Easy Infinitely Wide Networks with Neural Tangents

Friday, March 13, 2020

Posted by Samuel S. Schoenholz, Senior Research Scientist and Roman Novak, Research Engineer, Google Researchnatural language processingconversational agentsconnectomicseasierconverge toGaussian processesBayesian inferencegradient descent dynamicsconvolutional neural networkinfinitely wide networksuseful models in their own right

Left: A schematic showing how deep neural networks induce simple input / output maps as they become infinitely wide. Right: As the width of a neural network increases , we see that the distribution of outputs over different random instantiations of the network becomes Gaussian.

Neural Tangentsnew open-source software libraryJAXensemble.infinitely wideis described bycan be computed throughout trainingensemblesof these infinite-width networks be found here

In both plots we compare training of an ensemble of finite neural networks with the infinite-width ensemble of the same architecture. The empirical mean and variance of the finite ensemble is displayed as a dashed black line between two dotted black lines. The closed-form mean and variance of the infinite-width ensemble is displayed as a solid colored line inside a filled color region. In both plots finite- and infinite-width ensembles match very closely and can be hard to distinguish. Left: Outputs (vertical f-axis) on the input data (horizontal x-axis) as the training progresses. Right: Train and test loss with uncertainty over the course of training.

CIFAR-10wide residual networks

papertutorial Colab notebookGithub repoICLR 2020AcknowledgementsNeural Tangents is being actively developed by Lechao Xiao, Roman Novak, Jiri Hron, Jaehoon Lee, Alex Alemi, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. We also thank Yasaman Bahri and Greg Yang for the ongoing contributions to improve the library, as well as Sergey Ioffe, Ben Adlam, Ravid Ziv, and Jeffrey Pennington for frequent discussion and useful feedback. Finally, we thank Tom Small for creating the animation in the first figure.

Soli Radar-Based Perception and Interaction in Pixel 4

Thursday, March 12, 2020

Posted by Jaime Lien, Research Engineer and Nicholas Gillian, Software Engineer, Google Advanced Technology and ProjectsMotion Senseinteract with their Pixelface unlockSoliradarDesigning the Soli Radar System for Motion Sensealmost a centuryfrom the ground upbandwidthbeamwidth

Soli’s location in Pixel 4.

frequency-modulatedDoppler frequency

Left: Presence - Person walking towards the device. Middle: Reach - Person reaching towards the device. Right: Swipe - Person swiping over the device.

Designing Machine Learning Algorithms for RadarTensorFlowdigital signal processorTaking Soli from Concept to ProductRFIC

Hardware innovations included iteratively shrinking the radar system from a desktop-sized prototype to a single 5.0 mm x 6.5 mm RFIC, including antennas on package.

Vibration due to audio on Pixel 4 appearing as an artifact in Soli’s range-doppler signal representation.

Future DirectionsAcknowledgmentsThe work described above was a collaborative effort between Google Advanced Technology and Projects (ATAP) and the Pixel and Android product teams. We particularly thank Patrick Amihood for major contributions to this blog post.

Real-Time 3D Object Detection on Mobile Devices with MediaPipe

Wednesday, March 11, 2020

Posted by Adel Ahmadyan and Tingbo Hou, Software Engineers, Google Research 2D object predictionMediaPipe

3D Object Detection from a single image. MediaPipe Objectron determines the position, orientation and size of everyday objects in real-time on mobile devices.

Obtaining Real-World 3D Training DataLIDARaugmented realityARCoreARKithundreds of millions3D point clouds

Real-world data annotation for 3D object detection. Right: 3D bounding boxes are annotated in the 3D world with detected surfaces and point clouds. Left: Projections of annotated 3D bounding boxes are overlaid on top of video frames making it easy to validate the annotation.

AR Synthetic Data Generation

An example of AR synthetic data generation. The virtual white-brown cereal box is rendered into the real scene, next to the real blue book.

An ML Pipeline for 3D Object Detectionbuilt a single-stage modelMobileNetv2EPnPa prioriAdreno 650 mobile GPU

Network architecture and post-processing for 3D object detection.

Sample results of our network — [left] original 2D image with estimated bounding boxes, [middle] object detection by Gaussian distribution, [right] predicted segmentation mask.

Detection and Tracking in MediaPipe2D object detection and trackinginstant motion trackingMotion StillsMediaPipeend-to-end demo mobile applicationshoeschairs

Examples of our 3D object detection in the wild.

AcknowledgementsThe research described in this post was done by Adel Ahmadyan, Tingbo Hou, Jianing Wei, Matthias Grundmann, Liangkai Zhang, Jiuqiang Tang, Chris McClanahan, Tyler Mullen, Buck Bourdon, Esha Uboweja, Mogan Shieh, Siarhei Kazakou, Ming Guang Yong, Chuo-Ling Chang, and James Bruce. We thank Aliaksandr Shyrokau and the annotation team for their diligence to high quality annotations.