Google AI Blog: February 2019

Introducing PlaNet: A Deep Planning Network for Reinforcement Learning

Friday, February 15, 2019

Posted by Danijar Hafner, Student Researcher, Google AIartificial agentsreinforcement learningModel-free directlyDeepMind's DQNcontrolrobotsModel-based RLplan ahead, AlphaGounknowndynamicsDeepMindDeep Planning Network (PlaNet)source code

The PlaNet agent learning to solve a variety of continuous control tasks from images in 2000 attempts. Previous agents that do not learn a model of the environment often require 50 times as many attempts to reach comparable performance.

How PlaNet Worksplan over imageslatent dynamics model

Learned Latent Dynamics Model: In a latent dynamics model, the information of the input images is integrated into the hidden states (green) using the encoder network (grey trapezoids). The hidden state is then projected forward in time to predict future images (blue trapezoids) and rewards (blue rectangle).

A Recurrent State Space Model: A latent dynamics model with both deterministic and stochastic components, allowing to predict a variety of possible futures as needed for robust planning, while remembering information over many time steps. Our experiments indicate both components to be crucial for high planning performance.

A Latent Overshooting Objective: We generalize the standard training objective for latent dynamics models to train multi-step predictions, by enforcing consistency between one-step and multi-step predictions in latent space. This yields a fast and effective objective that improves long-term predictions and is compatible with any latent sequence model.

Planning in Latent Space: For planning, we encode past images (gray trapezoid) into the current hidden state (green). From there, we efficiently predict future rewards for multiple action sequences. Note how the expensive image decoder (blue trapezoid) from the previous figure is gone. We then execute the first action of the best sequence found (red box).

world modelsonline research paperPDF versionPlaNet vs. Model-Free Methods

A cartpole swing-up task, with a fixed camera, so the cart can move out of sight. The agent thus must absorb and remember information over multiple frames.

A finger spin task that requires predicting two separate objects, as well as the interactions between them.

A cheetah running task that includes contacts with the ground that are difficult to predict precisely, calling for a model that can predict multiple possible futures.

A cup task, which only provides a sparse reward signal once a ball is caught. This demands accurate predictions far into the future to plan a precise sequence of actions.

A walker task, in which a simulated robot starts off by lying on the ground, and must first learn to stand up and then walk.

PlaNet agents trained on a variety of image-based control tasks. The animation shows the input images as the agent is solving the tasks. The tasks pose different challenges: partial observability, contacts with the ground, sparse rewards for catching a ball, and controlling a challenging bipedal robot.

A3CD4PGDeepMind Control Suite

One Agent for All Tasks

Video predictions of the PlaNet agent trained on multiple tasks. Holdout episodes collected with the trained agent are shown above and open-loop agent hallucinations below. The agent observes the first 5 frames as context to infer the task and state and accurately predicts ahead for 50 steps given a sequence of actions.

Conclusion3D environmentsTPUsAcknowledgementsThis project is a collaboration with Timothy Lillicrap, Ian Fischer, Ruben Villegas, Honglak Lee, David Ha and James Davidson. We further thank everybody who commented on our paper draft and provided feedback at any point throughout the project.

Using Global Localization to Improve Navigation

Monday, February 11, 2019

Posted by Tilman Reinhardt‎, Software Engineer, Google MapsWhere exactly am I, andwhich way is north?" global localizationVisual Positioning ServiceStreet View

Due to limitations with accuracy and orientation, guidance via GPS alone is limited in urban environments. Using VPS, Street View and machine learning, Global Localization can provide better context on where you are relative to where you're going.

Where GPS Falls ShortlocalizationGPS

GPS signals bouncing off facades in an urban environment.

A New Approach to LocalizationexpectEnter Street ViewStreet View

Features matched from multiple images.

Combining Global Localization with Augmented RealityARCore

Local GuidesGoogle Lens

Announcing the Second Workshop and Challenge on Learned Image Compression

Wednesday, February 6, 2019

Posted by Nick Johnston, Software Engineer, Machine Perceptionwe announcedWorkshop and Challenge on Learned Image Compression CVPR 201823 accepted workshop papersthe competitionmean opinion scoreBetter Portable Graphics

An example image from the 2018 test set, comparing the original image to BPG, JPEG and the results from nine competing teams. All the methods are better than JPEG in color reproduction and many of them are comparable to BPG in their ability to create legible text on the sign.

Workshop and Challenge on Learned Image CompressionCVPR 2019PSNRMS-SSIMhuman evaluated rating taskcompression.ccAcknowledgementsThis workshop is being jointly hosted by researchers at Google, Twitter and ETH Zürich. We'd like to thank: George Toderici (Google), Michele Covell (Google), Johannes Ballé (Google), Nick Johnston (Google), Eirikur Agustsson (Google), Wenzhe Shi (Twitter), Lucas Theis (Twitter), Radu Timofte (ETH Zürich), Fabian Mentzer (ETH Zürich) for their contributions.

Real-time Continuous Transcription with Live Transcribe

Monday, February 4, 2019

Posted by Sagar Savla, Product Manager, Machine PerceptionWorld Health Organizationestimatesautomatic speech recognitionGoogle's ASRautomated captions in Youtubepresentations in Slidesmultiple improvementsCARTPalantypistSTTRLive TranscribeGoogle CloudBuilding Live Transcribeour previous workAudioSetpublished VGGish modelUser ExperienceGallaudet Universityeven small projectors

Displaying confidence level of the transcription. Yellow is high confidence, green is medium and blue is low confidence. White is fresh text awaiting context before finalizing. On the left, the coloring is at a per-phrase level while on the right is at a per-word level.¹ Research found them to be distracting to the user without providing conversational value.

UX research in this spacecocktail party problemrelative

The loudness and noise indicator is made of two concentric circles. The inner brighter circle, indicating the noise floor, tells a deaf user how audibly noisy the current environment is. The outer circle shows how well the speaker’s voice is received.Together, the circles visually show the relative difference intuitively.

Future Workon-device recognitionspeaker-separationspeech enhancementThe KeywordAcknowledgementsLive Transcribe was made by researchers Chet Gnegy, Dimitri Kanevsky, and Justin S. Paul in collaboration with Android Accessibility team members Brian Kemler, Thomas Lin, Alex Huang, Jacqueline Huang, Ben Chung, Richard Chang, I-ting Huang, Jessie Lin, Ausmus Chang, Weiwei Wei, Melissa Barnhart and Bingying Xia. We'd also like to thank our close partners from Gallaudet University, Christian Vogler, Norman Williams and Paula Tucker.
1 Eagle-eyed readers can see the phrase level confidence mode in use by Dr. Obeidat in the video above.^↩

Blog

Introducing PlaNet: A Deep Planning Network for Reinforcement Learning

Using Global Localization to Improve Navigation

Announcing the Second Workshop and Challenge on Learned Image Compression

Real-time Continuous Transcription with Live Transcribe

Labels

Archive

Feed