Blog
The latest news from Google AI
Soft Actor-Critic: Deep Reinforcement Learning for Robotics
Friday, January 18, 2019
Posted by Tuomas Haarnoja, Student Researcher and Sergey Levine, Faculty Advisor, Robotics at Google
Deep reinforcement learning (RL) provides the promise of fully automated learning of robotic behaviors directly from experience and interaction in the real world, due to its ability to process complex sensory input using general-purpose neural network representations. However, many existing RL algorithms require days or weeks (or more) worth of real-world data in order to converge to the desired behavior. Furthermore, such systems can be tough to deploy on complex robotic systems (such as legged robots) which can easily get damaged during the exploration phase, hyperparameter settings can be challenging to tune, and various safety considerations can introduce further limitations.
In collaboration with UC Berkeley,
we recently released Soft Actor-Critic
(SAC), a stable and efficient deep RL algorithm suitable for real-world robotic skill learning that is well-aligned with the requirements of robotic experimentation. Importantly, SAC is efficient enough to solve real-world robot tasks in only a handful of hours, and works on a variety of environments with a single set of hyperparameters. Below, we discuss some of the research behind SAC, and also describe some of our recent experiments.
Requirements for Real-World Robotic Learning
Real-world robotic experimentation brings significant challenges, such as constant interruptions in the data stream due to hardware failures and manual resets, and smooth exploration to avoid mechanical wear and tear on the robot, which set additional restrictions to both the algorithm and its implementation, including (but not limited to):
Good sample efficiency to lower the learning time
Minimal number of hyperparameters that require tuning
Reusing already collected data on different scenarios (known as off-policy learning)
Ensuring that learning and exploration does not damage the hardware
Soft Actor-Critic
Soft actor-critic is based on maximum entropy reinforcement learning, a framework that aims to both maximize the expected reward (which is the standard RL objective) and to maximize the policy's
entropy
. Policies with higher entropy are more random, which intuitively means that maximum entropy reinforcement learning prefers the most random policy that still achieves a high reward.
Why might this be desirable for robotic learning? The most obvious reason is that policies optimized for maximum entropy will be more robust: if the policy can tolerate highly random behavior during training, it is more likely to respond successfully to unexpected perturbations at test time. However, a more subtle reason is that training for maximum entropy can improve both the algorithm's robustness to hyperparameters and its sample efficiency (to learn more, see this
BAIR blog post
, and this
tutorial
).
Soft actor-critic maximizes the entropy augmented reward by learning a stochastic policy that maps states to actions and a
Q-function
that estimates the objective value of the current policy, optimizing them using approximate dynamic programming. In doing so, SAC views the objective as a grounded way to derive better reinforcement learning algorithms that perform consistently and are sample efficient enough to be applicable to real-world robotic applications. For technical details please see our
technical report
.
Performance of SAC
We evaluated SAC with two tasks: 1) quadrupedal walking with the Minitaur robot from
Ghost Robotics
, and 2) rotating a valve with a three finger
Dynamixel Claw
. Learning to walk presents a substantial challenge, as the robot is underactuated, and must therefore delicately balance contact forces on the legs to make forward progress. An untrained policy can lose balance and fall, and too many falls will eventually damage the robot, making sample-efficient learning essential.
Although we trained our policy only on flat terrain, we subsequently tested it on varied terrains and obstacles. In principle, policies learned with soft actor-critic should be robust to test-time perturbations, because they are trained to maximize entropy (i.e., inject maximal noise) at training-time. Indeed, we observe that the policies learned with our method are robust to these perturbations without any additional learning.
Illustration of learned walking, using SAC implemented on the Minitaur robot. A full video of the learning process can be found at our
project website
.
The manipulation task requires the hand to rotate a valve-like object so that the colored peg faces to the right, as shown below. This task is exceptionally challenging due to both the perception challenges and the need to control a hand with 9 degrees of freedom. In order to perceive the valve, the robot must use raw RGB images shown in the inset at the bottom right. The initial position of the valve is reset uniformly at random for each episode, forcing the policy to learn to use the raw RGB images to perceive the current valve orientation.
Soft actor-critic solves both of these tasks quickly: the Minitaur locomotion takes 2 hours, and the valve-turning task from image observations takes 20 hours. We also learned a policy for the valve-turning task without images by providing the actual valve position as an observation to the policy. Soft actor-critic can learn this easier version of the valve task in 3 hours. For comparison,
prior work
has used natural policy gradients to learn the same task without images in 7.4 hours.
Conclusion
Our work demonstrates that deep reinforcement learning based on maximum entropy framework can be applied to learn robot skills in challenging real-world settings. Since the policies are learned directly in the real world, they exhibit robustness to variations in the environment, which can be difficult to obtain otherwise. We also showed that we can learn directly from high-dimensional image observations, which represents a significant challenge in classical robotics. We hope that the release of SAC helps other research teams in their effort to adopt deep RL for more complex real-world tasks in the future.
For more technical details, please visit the
BAIR blog post
, or read an
early preprint of the locomotion experiment
and a
more complete description of the algorithm
. You can find the implementation on
GitHub
.
Acknowledgements
This research was done in collaboration between Google and UC Berkeley. We would like to thank all the people who were involved, including Sehoon Ha, Kristian Hartikainen, Jie Tan, George Tucker, Vincent Vanhoucke and Aurick Zhou.
Looking Back at Google’s Research Efforts in 2018
Tuesday, January 15, 2019
Posted by Jeff Dean, Senior Fellow and Google AI Lead, on behalf of the entire Google Research Community
2018 was an exciting year for Google's research teams, with our work advancing technology in many ways, including fundamental computer science research results and publications, the application of our research to emerging areas new to Google (such as healthcare and robotics), open source software contributions and strong collaborations with Google product teams, all aimed at providing useful tools and services. Below, we highlight just some of our efforts from 2018, and we look forward to what will come in the new year. For a more comprehensive look, please see our
publications in 2018
.
Ethical Principles and AI
Over the past few years, we have observed major advances in AI and the positive impact it can have on our products and the everyday lives of our billions of users. For those of us working in this field, we care deeply that AI is a force for good in the world, and that it is applied ethically, and to problems that are beneficial to society. This year we published the
Google AI Principles
, supported with a set of
responsible AI practices
outlining technical recommendations for implementation. In combination they provide a framework for us to
evaluate our own development of AI
, and we hope that other organizations can also use these principles to help shape their own thinking. It's important to note that because this field is evolving quite rapidly, best practices in some of the principles noted, such as "
Avoid creating or reinforcing unfair bias
" or "
Be accountable to people
", are also changing and improving as we and others conduct new research in areas like
ML fairness
and
model interpretability
. This research in turn leads to advances in our products to make them more inclusive and less biased, such as our work on
reducing gender biases
in Google Translate, and allows the
exploration
and
release
of more inclusive image datasets and models that enable computer vision to work for the diversity of global cultures. Furthermore, this work allows us to share best practices with the broader research community with the
Fairness Module
in the
Machine Learning Crash Course
.
AI for Social Good
The potential of AI to make dramatic impacts on many areas of social and societal importance is clear. One example of how AI can be applied to real-world problems is our
work on flood prediction
. In collaboration with many teams across Google, this research aims to provide accurate and timely fine-grained information about the likely extent and scope of flooding, enabling those in flood-prone regions to make better decisions about how best to protect themselves and their property.
A second example is our work on
earthquake aftershock prediction
, where we showed that a machine learning (ML) model can predict aftershock locations much more accurately than traditional physics-based models. Perhaps more importantly, because the ML model was designed to be interpretable, scientists have been able to make new discoveries about the behavior of aftershocks, leading to not only more accurate predictions, but also new levels of understanding.
We have also seen a huge number of external parties, sometimes in collaboration with Google researchers and engineers, using open source software like
TensorFlow
to tackle a wide range of scientific and social problems, such as
using convolutional neural networks to identify humpback whale calls
,
detecting new exoplanets
,
identifying diseased cassava plants
and more.
To spur creative activity in this area, we announced the
Google AI for Social Impact Challenge
in collaboration with
Google.org
, whereby individuals and organizations can receive grants from a total of $25M of funding, along with mentorship and advice from Google research scientists, engineers and other experts as they work to take a project with large potential social impact from idea to reality.
Assistive Technology
Much of our research centered on using ML and computer science to help our users accomplish things faster and more effectively. Often, these results in collaborations with various product teams to release the fruits of this research in various product features and settings. One example is
Google Duplex
, a system that requires research in natural language and dialogue understanding, speech recognition, text-to-speech, user understanding and effective UI design to all come together to enable an experience whereby a user can say "
Can you book me a haircut at 4 PM today?
", and a virtual agent will interact on your behalf over the telephone to handle the necessary details.
Other examples include
Smart Compose
, a tool that uses predictive models to give relevant suggestions about how to compose emails, making the process of email composition faster and easier, and
Sound Search
, a technology built on the
Now Playing
feature that enables you to discover what song is playing fast and accurately. Additionally,
Smart Linkify
in Android shows how we can use an on-device ML model to make many different kinds of text that appear on the screen of your phone more useful by understanding the kind of text you're selecting (e.g. knowing that something is an address, so we can offer a shortcut to a maps or direction link).
An important focus in our research is helping to make products like the Google Assistant
support more languages
and allow
better understanding of semantic similarity
, even when very different ways of expressing the same concept or idea are used. Underlying new product capabilities like these is research we performed on improving the quality of both
speech synthesis
and
text-to-speech for languages without much training data available
.
Quantum computing
Quantum computing is an emerging paradigm for computing that promises the ability to solve challenging problems that no classical computer can solve. We have been actively pursuing research in this area for the past several years, and we believe the field is on the cusp of demonstrating this capability for at least one problem (so-called
quantum supremacy
), which will be a watershed event for the field. Over the last year we produced
a number of exciting new results
, including the development of
Bristlecone
, a new 72-qubit quantum computing device, which scales the size of problems that can be tackled in quantum computers in the run-up towards quantum supremacy.
A Bristlecone chip being installed by Research Scientist Marissa Giustina at the Quantum AI Lab in Santa Barbara.
We also released
Cirq
, an open source programming framework for quantum computers, and explored how
quantum computers could be used for neural networks
. Finally, we shared our experience and techniques for
understanding performance fluctuations in quantum processors
, and shared some thoughts on how
quantum computers might be useful as a computational substrate for neural networks
. We're looking forward to exciting results in the quantum computing space in 2019!
Natural Language Understanding
Natural language research at Google had an exciting 2018, with a
mix of basic research as well as product-focused collaborations
. We developed improvements to our
Transformer work from 2017
, resulting in a new parallel-in-time version of the model called the
Universal Transformer
that shows strong gains across a number of natural language tasks including translation and linguistic reasoning. We also developed
BERT
, the first deeply bidirectional, unsupervised language representation, pre-trained using only a plain text corpus, that can then be fine-tuned on a wide variety of natural language tasks using transfer learning. BERT shows significant improvements over previous state-of-the-art results on 11 natural language tasks.
BERT also improves the state-of-the-art by 7.6% absolute on the very challenging
GLUE benchmark
, a set of 9 diverse Natural Language Understanding (NLU) tasks.
In addition to collaborating with various research teams to enable Smart Compose and Duplex (discussed previously), we worked to make the Google Assistant
handle multilingual use cases better, with the goal of making
the Assistant naturally conversational for
all
users.
Perception
Our perception research tackles the hard problems of allowing computers to understand images, sounds, music and video, as well as providing more powerful tools for image capture, compression, processing, creative expression, and augmented reality. In 2018, our technology improved Google Photos' ability to organize the content that users most care about, such as people and
pets
. Google Lens and the Assistant enabled users to
learn about the natural world
,
answer questions in real-time
, and
do more with Lens in Google Images
. A key aspect of the Google AI mission is to empower others to benefit from our technology, and we've made a lot of progress this year in improving capabilities and building blocks that are parts of Google APIs. Examples include improved and
new capabilities in vision and video in Cloud ML APIs
and
face-related on-device building blocks
through ML Kit.
Google Lens can help you learn more about the world around you. Here, Lens identifies the breed of this dog. Learn more in
this blog post
.
In 2018, our contributions to academic research included advances in deep learning for 3D scene understanding, such as
stereo magnification
, which enables synthesizing novel photorealistic views of a scene. Our ongoing research on better understanding images and video enables users to find, organize, enhance and improve images and video in Google products such as Photos, YouTube, Search and more. In 2018, notable advances included
a fast bottom-up model for joint pose estimation and person instance segmentation
, a
system for visualizing complex motion
, a
system
which models spatio-temporal relations between people and objects and improvements in video action recognition based on
distillation
and
3D convolutions
.
In the audio domain, we proposed a method for
unsupervised learning of semantic audio representations
as well as significant improvements to
expressive and human-like speech synthesis
. Multimodal perception is an increasingly important research topic.
Looking to Listen
combines visual and auditory cues in an input video to isolate and enhance the speech of desired speakers in a video. This technology could support a range of applications, from speech enhancement and recognition in videos, through video conferencing, to improved hearing aids, especially in situations where multiple people are speaking.
Enabling perception on resource-constrained platforms has becoming increasingly important.
MobileNetV2
is Google's
next-generation mobile computer vision model
and our MobileNets are used widely across academia and industry.
MorphNet
proposes an efficient method for learning the structure of deep networks that results in across-the-board performance improvements on image and audio models while respecting computational resource constraints, and more recent work on
automatic generation of mobile network architectures
demonstrates that even higher performance is possible.
Computational Photography
The improvements in quality and versatility of cell phone cameras over the last few years has been nothing short of remarkable. A modest part of this is improvements in the actual physical sensors used in phones, but a much greater part of it is due to advances in the scientific field of computational photography. Our research teams publish their new research techniques, and work closely with the Android and Consumer Hardware teams at Google to deliver this research into your hands in the latest Pixel and Android phones and other devices. In 2014, we introduced
HDR+
, a technique whereby the camera captures a burst of frames, aligns the frames in software, and merges them together with computational software. Originally in the HDR+ work, this was to enable pictures to have higher dynamic range than was possible with a single exposure. However, capturing a burst of frames and then performing computational analysis of these frames is a general approach that has enabled many advances in cameras in 2018. For example, it allowed the development of
Motion Photos in Pixel 2
and the
Augmented Reality mode in Motion Stills
.
Motion photos on the Pixel 2 in Google Photos. For more examples, check out this
Google Photos album
.
Augmented chicken family with Motion Stills AR mode.
This year, one of our primary efforts in computational photography research was to create a new capability called
Night Sight
, which enables Pixel phone cameras to "see in the dark", earning praise by both
press
and
users
. Of course, Night Sight is just one of the new software-enabled camera features our teams have developed to help you take the perfect photo, including
using ML to provide better portrait mode shots
, seeing better and further with
Super Res Zoom
and capturing special moments with
Top Shot
and
Google Clips
.
Left: iPhone XS (full resolution image
here
). Right: Pixel 3 Night Sight (full resolution image
here
).
Algorithms and Theory
Algorithms are the backbone of Google systems and touch all our products, from
routing algorithms behind Google trips
to
consistent hashing for Google cloud
. Over the past year, we continued our research in
algorithms and theory
covering a wide range of areas from theoretical foundations to applied algorithms, and from
graph mining
to privacy-preserving computation. Our work in optimization spans areas from studying continuous optimization for machine learning to distributed combinatorial optimization. In the former area,
our work on studying convergence of stochastic optimization algorithms for training neural networks
(which won an
ICLR 2018
Best Paper Award) exhibited issues with popular gradient-based optimization methods (such as some variants of
ADAM
), but provided a solid foundation for new gradient-based optimization methods.
Performance comparison of ADAM and AMSGRAD on a synthetic example of a simple one dimensional convex problem inspired by our examples of non-convergence. The first two plots (left and center) are for the online setting and the the last one (right) is for the stochastic setting.
In
distributed optimization
, we worked to improve the round and communication complexity of well-studied combinatorial optimization problems such as matchings in graphs
via round compression
and
via core-sets
, as well as
submodular maximization
, and
k-core decomposition
. On the more applied side, we developed algorithmic techniques for solving
set cover at scale via sketching
and for solving
balanced partitioning and hierarchical clustering
for graphs with trillions of edges. Our work on
online delivery services
was nominated for the best paper award at
WWW'18
. Finally, our open source optimization
OR-tools platform
won 4 gold medals at the
2018 Minizinc
constraint programming competition.
In algorithmic choice theory, we have proposed
new models
and investigated the problems of
reconstruction
and
learning a mixture of multinomial logits
. We also studied the classes of functions
learnable by neural networks
and how to use machine-learned oracles to
improve classic
online algorithms
.
Understanding learning techniques with strong privacy guarantees is of great importance for us at Google. In this context, we developed two new means of analyzing how differential privacy can be amplified
by iteration
and
by shuffling
. We also applied differential privacy techniques to design
incentive-aware learning
methods that are robust against gaming. Such learning techniques have applications in efficient online market design. Our
new research
in the area of
market algorithms
include also techniques to help advertisers
test incentive compatibility
of ad auctions, and
optimizing ad refresh for in-app advertising
. We also pushed the boundaries of state-of-the-art dynamic mechanisms for repeated auctions, and presented dynamic auctions that are robust against
lack of prediction of future
, against
noisy forecasts
, or against
heterogenous buyer behaviour
, and extend our results to
dynamic double auctions
. Finally, in the context of robustness in online optimization and online learning, we developed
new online allocation algorithms for stochastic input with traffic spikes
and
new bandit algorithms robust to corrupted data
.
Software Systems
A large part of our research on software systems continues to relate to building machine-learning models and to TensorFlow in particular. For example, we published on the design and implementation of
dynamic control flow
for TensorFlow 1.0. Some of our newer research introduces a system that we call
Mesh TensorFlow
, which makes it easy to specify large-scale distributed computations with model parallelism, sometimes with billions of parameters. As another example, we
released
a library for scalable deep neural ranking using TensorFlow.
The TF-Ranking library supports multi-item scoring architecture, an extension of traditional single-item scoring.
We also released
JAX
, an accelerator-backed variant of NumPy that supports automatic differentiation of Python functions to arbitrary order. While JAX is not part of TensorFlow, it leverages some of the same underlying software infrastructure (e.g.
XLA
), and some of its ideas and algorithms have been helpful to our TensorFlow projects. Finally, we continued our research on the security and privacy of machine learning, and our development of open source frameworks for safety and privacy in AI systems, such as
CleverHans
and
TensorFlow Privacy
.
Another important research direction for us is the application of ML to software systems, at many levels of the stack. For instance, we continued work on
placement of computations onto devices
, with a hierarchical model, and we contributed to
learning memory access patterns
. We also continued to explore how
learned indices
could be used to replace traditional index structures in database systems and storage systems. As I wrote last year, we believe that we are just scratching the surface in terms of the use of machine learning in computer systems.
The
Hierarchical Planner's
placement of a NMT (4-layer) model. White denotes CPU and the four colors each represent one of the GPUs. Note that every step of every layer is allocated across multiple GPUs. This placement is 53.7% faster than that generated by a human expert.
In 2018 we learned about
Spectre
and
Meltdown
, new classes of serious security vulnerabilities in modern computer processors, thanks to Google's
Project Zero
team in collaboration with others. These and related vulnerabilities will keep computer architecture researchers quite busy. In our continuing efforts to model CPU behavior, our Compiler Research team integrated their
tool
for measuring machine instruction latency and port pressure into LLVM, making possible better compilation decisions.
Google products, our Cloud offerings and inference for machine learning models depend critically on the ability to provide large-scale, reliable, efficient technical infrastructure for computing, storage and networking. A few research highlights from the past year include
the evolution of Google's Software Defined Networking WAN
, a
stand-alone, federated query processing platform
that executes SQL queries against data stored in different file-based formats, in many storage systems (BigTable, Spanner, Google Spreadsheets, etc.) and
a report on our extensive use of code review
, investigating the motivations behind code review at Google, current practices, and developers' satisfaction and challenges.
Running a large-scale web service such as content hosting, requires load balancing with stability in a
dynamic
environment. We developed a
consistent hashing scheme
with
tight provable guarantees
on the maximum load of each server, and
deployed it
for our cloud customers in
Google Cloud Pub/Sub
. After making an
earlier version of our paper
available, engineers at
Vimeo
found the paper, implemented and open sourced it in
haproxy
, and used it for their load balancing project at Vimeo. The results were dramatic: applying these algorithmic ideas helped them decrease the cache bandwidth by a factor of
almost 8
, eliminating a scaling bottleneck.
AutoML
AutoML, also known as meta-learning, is the use of machine learning to automate some aspects of machine learning. We have been performing research in this space for many years, and the long-term goal is to develop learning systems that can learn to take a new problem and solve it automatically, using insights and capabilities derived from other problems that have been previously solved. Our earlier work in this space has mostly used reinforcement learning, but we are also interested in the use of evolutionary algorithms. Last year we showed how
evolutionary algorithms can be used to automatically discover state-of-the-art neural network architectures
for a variety of visual tasks. We also explored how reinforcement learning can be applied to other problems than just neural network architecture search, showing that it can be used to 1)
automatically generate image transformation sequences
that improve the accuracy of a wide variety of image models, and 2)
find new symbolic optimization expressions that are more effective than the commonly used optimization update rules
. Our work on
AdaNet showed how to have a fast and flexible AutoML algorithm with learning guarantees
.
AdaNet adaptively growing an ensemble of neural networks. At each iteration, it measures the ensemble loss for each candidate, and selects the best one to move onto the next iteration.
Another focus for us was on automatically discovering neural network architectures that are computationally efficient, so that they can run in environments such as mobile phones or autonomous vehicles that have tight constraints on either computational resources or on inference time. For this, we showed that
combining the accuracy of a model with its inference computation time in the reward function for a reinforcement learning architecture search
can find models that are highly accurate while meeting particular performance constraints. We also
explored using ML to learn to automatically compress ML models
to have fewer parameters and use less computational resources.
TPUs
Tensor Processing Units
(TPUs) are Google's internally-developed ML hardware accelerators, designed from the ground up to power both training and inference at scale. TPUs have enabled Google research breakthroughs such as
BERT
(discussed previously), and they also allow researchers around the world to build on Google research via open source and to pursue new breakthroughs of their own. For example, anyone can fine-tune BERT on TPUs for free
via Colab
, and the
TensorFlow Research Cloud
has given thousands of researchers the opportunity to benefit from even larger amounts of free Cloud TPU computing power. We've also made multiple generations of TPU hardware commercially available as
Cloud TPUs
, including ML supercomputers called Cloud TPU Pods that make
large-scale ML training
much more accessible. Internally, in addition to enabling faster advances in ML research, TPUs have driven major improvements across Google's core products, including Search, YouTube, Gmail, Google Assistant, Google Translate, and many others. We look forward to seeing ML teams both here at Google and elsewhere achieve even more with ML via the unprecedented computing scale that TPUs provide.
An individual TPU v3 device (left) and a portion of a TPU v3 Pod (right). TPU v3 is the latest generation of Google's Tensor Processing Unit (TPU) hardware. Available to external customers as
Cloud TPU v3
, these systems are liquid-cooled for maximum performance (computer chips + liquid = exciting!), and a full TPU v3 Pod can apply more than 100 petaflops of computational power to the world's largest ML problems.
Open Source Software and Datasets
Releasing open source software and the creation of new public datasets are two major ways that we contribute to the research and software engineering communities. One of our largest efforts in this space is
TensorFlow
, a widely popular system for ML computations that we
released in November 2015
. We celebrated TensorFlow's third birthday in 2018, and during this time, TensorFlow has been downloaded more than 30M times, with over 1700 contributors adding 45,000 commits. In 2018, TensorFlow had eight major releases and added major capabilities such as eager execution and distribution strategies. We launched public design reviews engaging the community in the development process, and we engaged contributors via special interest groups. With the launches of associated products such as
TensorFlow Lite
,
TensorFlow.js
and
TensorFlow Probability
, the TensorFlow ecosystem grew dramatically in 2018.
We are happy that TensorFlow has the strongest Github user retention of the top machine learning and deep learning frameworks. The TensorFlow team is also working to address Github issues faster and provide a smooth path for external contributors. In research, we continue to power much of the world's machine learning and deep learning research on a published paper basis according to Google Scholar data. TensorFlow Lite is now on more than 1.5B devices globally after being available for just one year. Additionally, TensorFlow.js is the number one ML framework for JavaScript; in the nine months since launch, it had over 2M
Content Delivery Network
(CDN) hits, 250K downloads and more than
10,000 stars on Github
.
In addition to continued work on existing open source ecosystems, in 2018 we
introduced a new framework for flexible and reproducible reinforcement learning
,
new visualization tools to rapidly understand the characteristics of a dataset
(without needing to write any code), added
a high-level library for expressing machine learning problems that involve learning-to-rank
(the process of ordering a list of items in a way that maximizes the utility of the entire list, applicable across domains that include search engines, recommender systems, machine translation, dialogue systems and even computational biology), released
a framework for fast and flexible AutoML solutions with learning guarantees
, a
library for doing in-browser realtime t-SNE visualizations
using TensorFlow.js and added
FHIR tools and software for working with electronic healthcare data
(discussed in the healthcare section of this post).
Real-time evolution of the tSNE embedding for the complete MNIST dataset. The dataset contains images of 60,000 handwritten digits.
You can find a live demo here
.
Public datasets are often a great source of inspiration that lead to great progress across many fields, since they give the broader community both access to interesting data and problems as well as a healthy competitive drive to achieve better results on a variety of tasks. This year we were happy to release
Google Dataset Search
, a new tool for finding public datasets from all of the web. Over the years we have also
curated and released many new, novel datasets
, including everything from millions of general annotated images or videos, to a crowd-source Bengali dataset for speech recognition to robot arm grasping datasets and more. In 2018, we added even more datasets to that list.
Pictures from India & Singapore added to
Open Images Extended
using the Crowdsource app.
We released
Open Images V4
, a dataset containing 15.4M bounding-boxes for 600 categories on 1.9M images, as well as 30.1M human-verified image-level labels from 19,794 categories. We also
extended this dataset to add more diversity of people and scenes from all over the world
, by adding 5.5M generated annotations provided by tens of thousands of users from all over the world using
crowdsource.google.com
. We released the Atomic Visual Actions (
AVA
) dataset that provides audiovisual annotations of video for improving the state of the art in understanding
human actions
and
speech in video.
We also
announced an updated YouTube-8M, and the 2nd YouTube-8M Large-Scale Video Understanding Challenge and Workshop
. The
HDR+ Burst Photography Dataset
aims to enable a wide variety of research in the field of computational photography, and
Google-Landmarks
was a new dataset and challenge for landmark recognition. And while not a dataset release, we explored techniques that can enable faster creation of visual datasets using
Fluid Annotation
, an exploratory ML-powered interface for faster image annotation.
Visualization of the fluid annotation interface in action on image from COCO dataset. Image credit:
gamene
,
original image
.
From time-to-time, we also help establish new kinds of challenges for the research community, so that we can all work together on solving difficult research problems. Often these are done with the release of a new dataset, but not always. This year, we established new challenges around the
Inclusive Images Challenge
, to work towards making more robust models that are free from many kinds of biases, the
iNaturalist 2018 Challenge
which aims to enable computers' fine-grained discrimination of visual categories (such as species of plants in an image), a
Kaggle "Quick, Draw!" Doodle Recognition Challenge
to create a better classifier for the QuickDraw challenge game, and
Conceptual Captions
, a larger-scale image captioning dataset and challenge aimed at enabling better image captioning model research.
Robotics
In 2018, we made significant progress towards our goal of understanding how ML can teach robots how to act in the world, achieving a new milestone in the ability to
teach robots to grasp novel objects
(best systems paper at CoRL'18), and using it
to learn about objects without human supervision
. We've also made progress in learning robot motion by
combining ML and sampling-based methods
(best paper in service robotics at
ICRA'18
) and
learning robot geometry
for faster planning. We've made great strides in our ability to better
perceive the structure of the world
from autonomous observation. For the first time, we've been able to successfully train
deep reinforcement learning models online on real robots
, and are finding new, theoretically grounded ways, to
learn stable approaches
to robot control.
Applications of AI to Other Fields
In 2018, we have applied ML to a wide variety of problems in the physical and biological sciences. Using ML, we can supply scientists with the equivalent of hundreds or thousands of research assistants digging through data, which then frees the scientists to become more creative and productive.
Our
Nature Methods
paper on
high-precision automated reconstruction of neurons
proposed a new model that
improves the accuracy of automated interpretation of connectomics data by an order of magnitude
over previous deep learning techniques.
Our algorithm in action as it traces a single neurite in 3d in a songbird brain.
Some other examples of applying ML to science include:
Finding new planets outside our solar system by data mining light curves of stars
Recognizing the origin or function of short DNA sequences
Automatically detecting out-of-focus microscope images
Digitally creating images of the same cells with multiple stains
Automatically mapping mass spectrometry output to peptide chains
A pre-trained TensorFlow model rates focus quality for a montage of microscope image patches of cells in
Fiji (ImageJ)
. Hue and lightness of the borders denote predicted focus quality and prediction uncertainty, respectively.
Health
For the past several years, we have been
applying ML to health
, an area that affects every one of us, and is also one where we believe ML can make a tremendous difference by augmenting the intuitions and experience of healthcare professionals. Our general approach in this space is to collaborate with healthcare organizations to tackle basic research problems (using feedback from clinical experts to make our results more robust), and then publish the results in well-respected, peer-reviewed scientific and clinical journals. Once the research has been clinically and scientifically validated, we then conduct user and HCI research to understand how we can deploy this in real-world clinical settings. In 2018, we expanded our efforts across the broad space of computer-aided diagnostics to clinical task predictions as well.
At the end of 2016, we
published work
showing that a model trained to assess retinal fundus images for signs of diabetic retinopathy was able to perform on-par to slightly-better than U.S. medical-board-certified ophthalmologists at this task in a retrospective study. In 2018, we were able to show that by having the training images labeled by retinal specialists and by using an adjudicated protocol (where multiple retinal specialists convene and have to arrive at a single collective assessment for each fundus image), we could arrive at a model that
is on-par with retinal specialists
. Later, we published an evaluation that showed how
pairing ophthalmologists and this ML model allow them to make more accurate decisions
than either alone. We have deployed this diabetic retinopathy detection system in partnership with our Alphabet colleagues at
Verily
at over 10 sites including
Aravind Eye Hospitals
in India and at
Rajavithi Hospital affiliated with the Ministry of Health in Thailand
.
On the left is a retinal fundus image graded as having moderate DR ("Mo") by an adjudication panel of ophthalmologists (ground truth). On the top right is an illustration of the predicted scores ("N" = no DR, "Mi" = Mild DR, "Mo" = Moderate DR) from the model. On the bottom right is the set of scores given by physicians without assistance ("Unassisted") and those who saw the model's predictions ("Grades Only").
In work that
medical and eye specialists found quite remarkable
, we also published research on a machine learning model that
can assess cardiovascular risk from retinal images
. This shows early promising signs for a novel, non-invasive biomarker that can help clinicians better understand the health of their patients.
We have also continued our focus on pathology this year, showing
how to improve the grading of prostate cancer using ML
,
detect metastatic breast cancer with deep learning
, and developed a
prototype for an augmented-reality microscope
that can aid pathologists and other scientists by overlaying visual information derived from computer vision models into the visual field of the microscopist in real time.
For the past four years, we have had a significant research effort around using deep learning on electronic health records to make clinically-relevant predictions. In 2018, in collaboration with
University of Chicago Medicine
,
UCSF
and
Stanford Medicine
, we
published work in Nature Digital Medicine
showing how ML models applied to de-identified electronic medical records can make significantly higher accuracy predictions for a variety of clinically relevant tasks than the current clinical best practice. As part of this work, we developed tools to make it significantly easier to create these models even on quite different tasks and quite different underlying EHR data sets. We have
open sourced software
related to the
Fast Healthcare Interoperability Resources
(FHIR) standard that we developed in this work to help make working with medical data easier and more standardized (see this
GitHub repository
). We also
improved the accuracy, speed and utility of our deep learning-based variant caller, DeepVariant
. The team has forged ahead with partners and recently published the peer-reviewed paper in
Nature Biotechnology
.
When applying ML to historically-collected data, it's important to understand the populations that have experienced human and structural biases in the past and how those biases have been codified in the data. Machine-learning offers an opportunity to detect and address bias and
to proactively advance health equity
, which we are designing our systems to do.
Research Outreach
We interact with the external research community in many different ways, including faculty engagement and student support. We are proud to host hundreds of undergraduate, M.S. and Ph.D. students as interns during the academic year, as well as
providing multi-year Ph.D. fellowships
to students throughout North America, Europe, and the Middle East. In addition to financial support, each of the fellowship recipients is assigned one or more Google researchers as a mentor, and we bring together all the fellows for an
annual Google Ph.D. Fellowship Summit
, where they are exposed to state-of-the-art research being pursued at Google and given the opportunity to network with Google's researchers as well as other PhD Fellows from around the world.
Complementing this fellowship program is the
Google AI Residency
, a way of allowing people who want to learn to conduct deep learning research to spend a year working alongside and being mentored by researchers at Google. Now in
its third year
, residents are embedded in various teams across Google's
global offices
, pursuing research in areas such as
machine learning
,
perception
,
algorithms and optimization
,
language understanding
,
healthcare
and
much more
. With applications having just closed for the fourth year of this program, we are excited to see the research the new cohort of residents will pursue in 2019.
Each year, we also support a number of faculty members and students on research projects through our
Google Faculty Research Awards program
. In 2018, we also continued to host workshops at Google locations for faculty and graduate students in particular areas, including a
workshop on AI/ML Research and Practice
hosted in our Bangalore, India office, an
Algorithms & Optimization Workshop
hosted in our Zürich office, a workshop on healthcare applications of ML hosted in Sunnyvale and a workshop on Fairness and Bias in ML hosted in our Cambridge, MA office.
We believe that contributing openly to the broader research community is a critical part of supporting a healthy and productive research ecosystem. In addition to our open source and dataset releases,
much of our research is published openly
in top conference venues and journals, and we actively participate in the organization and sponsorship of conferences, all across the spectrum of different disciplines. For just a small sample, see our involvement at
ICLR 2018
,
NAACL 2018
,
ICML 2018
,
CVPR 2018
,
NeurIPS 2018
,
ECCV 2018
and
EMNLP 2018
. Googlers also participated extensively in
ASPLOS
,
HPCA
,
ICSE
,
IEEE Security & Privacy
,
OSDI
,
SIGCOMM
, and many other conferences in 2018.
New Places, New Faces
In 2018, we were excited to welcome many new people with a wide range of backgrounds into our research organization. We
announced our first AI research office in Africa
, located in Accra, Ghana. We expanded our AI research presence in
Paris
,
Tokyo
and
Amsterdam
, and opened a research lab in
Princeton
. We continue to hire talented people into our offices all over the world, and you can learn more about joining our research efforts
here
.
Looking Forward to 2019
This blog post summarizes just a small fraction of the research performed in 2018. As we look back on 2018, we're excited (and proud!) of the breadth and depth of what we have accomplished. In 2019, we look forward to having even more impact on Google's direction and products, as well as on the broader research and engineering community!
Top Shot on Pixel 3
Thursday, December 20, 2018
Posted by Li Zhang and Wei (Alex) Hong, Software Engineers
Life is full of meaningful moments — from a child’s first step to an impromptu jump for joy — that one wishes could be preserved with a picture. However, because these moments are often unpredictable, missing that perfect shot is a frustrating problem that smartphone camera users face daily. Using our experience from developing
Google Clips
, we wondered if we could develop new techniques for the Pixel 3 camera that would allow everyone to capture the perfect shot every time.
Top Shot
is a new feature recently launched with
Pixel 3
that helps you to capture precious moments precisely and automatically at the press of the shutter button. Top Shot saves and analyzes the image frames before and after the shutter press on the device in real-time using computer vision techniques, and recommends several alternative high-quality
HDR+
photos.
Examples of Top Shot on Pixel 3. On the left, a better smiling shot is recommended. On the right, a better jump shot is recommended. The recommended images are high-quality HDR+ shots.
Capturing Multiple Moments
When a user opens the Pixel 3 Camera app, Top Shot is enabled by default, helping to capture the perfect moment by analyzing images taken both before and after the shutter press. Each image is analyzed for some qualitative features (e.g., whether the subject is smiling or not) in real-time and entirely on-device to preserve privacy and minimize latency. Each image is also associated with additional signals, such as
optical flow
of the image, exposure time, and gyro sensor data to form the input features used to score the frame quality.
When you press the shutter button, Top Shot captures up to 90 images from 1.5 seconds before and after the shutter press, selecting up to two alternative shots to save in high resolution — the original shutter frame and high-res alternatives for you to review (other lower-res frames can also be reviewed as desired). The shutter frame is processed and saved first. The best alternative shots are saved afterwards.
Google’s Visual Core
on Pixel 3 is used to process these top alternative shots as HDR+ images with a very small amount of extra latency, and are embedded into the file of the
Motion Photo
.
Top-level diagram of Top Shot capture.
Given Top Shot runs in the camera as a background process, it must have very low power consumption. As such, Top Shot uses a hardware-accelerated
MobileNet
-based
single shot detector
(SSD). The execution of such optimized models is also throttled by power and thermal limits.
Recognizing Top Moments
When we set out to understand how to enable people to capture the best moments with their camera, we focused on three key attributes: 1) functional qualities like lighting, 2) objective attributes (are the subject's eyes open? Are they smiling?), and 3) subjective qualities like emotional expressions. We designed a computer vision model to recognize these attributes while operating in a low-latency, on-device mode.
During our development process, we started with a vanilla MobileNet model and set out to optimize for Top Shot, arriving at a customized architecture that operated within our accuracy, latency and power tradeoff constraints. Our neural network design detects low-level visual attributes in early layers, like whether the subject is blurry, and then dedicates additional compute and parameters toward more complex objective attributes like whether the subject's eyes are open, and subjective attributes like whether there is an emotional expression of amusement or surprise. We trained our model using
knowledge distillation
over a large number of diverse face images using
quantization
during both training and inference.
We then adopted a layered
Generalized Additive Model
(GAM) to provide quality scores for faces and combine them into a weighted-average “frame faces” score. This model made it easy for us to interpret and identify the exact causes of success or failure, enabling rapid iteration to improve the quality and performance of our attributes model. The number of free parameters was on the order of dozens, so we could optimize these using Google's black box optimizer,
Vizier
, in tandem with any other parameters that affected selection quality.
Frame Scoring Model
While Top Shot prioritizes for face analysis, there are good moments in which faces are not the primary subject. To handle those use cases, we include the following additional scores in the overall frame quality score:
Subject motion saliency score
— the low-resolution optical flow between the current frame and the previous frame is estimated in ISP to determine if there is salient object motion in the scene.
Global motion blur score
— estimated from the camera motion and the exposure time. The camera motion is calculated from sensor data from the gyroscope and OIS (optical image stabilization).
“3A” scores
— the status of auto exposure, auto focus, and auto white balance, are also considered.
All the individual scores are used to train a model predicting an overall quality score, which matches the frame preference of human raters, to maximize end-to-end product quality.
End-to-End Quality and Fairness
Most of the above components are each evaluated for accuracy independently However, Top Shot presents requirements that are uniquely challenging since it’s running real-time in the Pixel Camera. Additionally, we needed to ensure that all these signals are combined in a system with favorable results. That means we need to gauge our predictions against what our users perceive as the “top shot.”
To test this, we collected data from hundreds of volunteers, along with their opinions of which frames (out of up to 90!) looked best. This donated dataset covers many typical use cases, e.g. portraits, selfies, actions, landscapes, etc.
Many of the 3-second clips provided by Top Shot had more than one good shot, so it was important for us to engineer our quality metrics to handle this. We used some modified versions of traditional
Precision and Recall
, some classic ranking metrics (such as
Mean Reciprocal Rank
), and a few others that were designed specifically for the Top Shot task as our objective. In addition to these metrics, we additionally investigated causes of image quality issues we saw during development, leading to improvements in avoiding blur, handling multiple faces better, and more. In doing so, we were able to steer the model towards a set of selections people were likely to rate highly.
Importantly, we tested the Top Shot system for fairness to make sure that our product can offer a consistent experience to a very wide range of users. We evaluated the accuracy of each signal used in Top Shot on several different subgroups of people (based on gender, age, ethnicity, etc), testing for accuracy of each signal across those subgroups.
Conclusion
Top Shot is just one example of how Google leverages optimized hardware and cutting-edge machine learning to provide useful tools and services. We hope you’ll find this feature useful, and we’re committed to further improving the capabilities of mobile phone photography!
Acknowledgements
This post reflects the work of a large group of Google engineers, research scientists, and others including: Ari Gilder, Aseem Agarwala, Brendan Jou, Chris Breithaupt, David Karam, Eric Penner, Farooq Ahmad, Henri Astre, Hillary Strickland, John Zhang, Marius Renn, Matt Bridges, Maxwell Collins, Navid Shiee, Ryan Gordon, Sarah Clinckemaillie, Shu Zhang, Vivek Kesarwani, Xuhui Jia, Yukun Zhu and Yuzo Watanabe.
Google AI Princeton: Current and Future Research
Tuesday, December 18, 2018
Posted by Elad Hazan and Yoram Singer, Research Scientists, Google AI and Princeton University
Google has long
partnered with academia
to advance research, collaborating with universities all over the world on joint research projects which result in novel developments in Computer Science, Engineering, and related fields. Today we announce the latest of these academic partnerships in the form of a new lab, across the street from Princeton University’s historic Nassau Hall, opening early next year. By fostering closer collaborations with faculty and students at Princeton, the lab aims to broaden research in multiple facets of machine learning, focusing its initial research efforts on optimization methods for
large-scale machine learning
,
control theory
and
reinforcement learning
. Below we give a brief overview of the research progress thus far.
Large-Scale Optimization
Imagine you have gone for a mountain hike and have run out of water. You need to get to a lake. How can you do so most efficiently? This is a matter of optimizing your route, and the mathematical analogue of this is the
gradient descent method
. You therefore move in the direction of steepest descent until you find the nearest lake at the bottom of your path. In the language of optimization, the location of the lake is referred to as a (local) minimum. The trajectory of gradient descent resembles the path, shown below, a thirsty yet avid hiker would take in order to get down to a lake as fast as she can.
Gradient descent (GD), and its randomized version,
stochastic gradient descent
(SGD), are the methods of choice for optimizing the weights of neural networks. Stacking all of the parameters together, we form a set of cells organized into vectors Let us take a simplistic view and assume that our neural net merely has 5 different parameters. Taking a gradient descent step amounts to subtracting the gradient vector (red) from the current set of parameters (blue) and putting the result back into the parameter vector.
Going back to our avid hiker, suppose she finds an unmarked path that is long and narrow, with limited visibility as she gazes down. If she follows the descent method her path would zig-zag down the hill, as shown in the illustration below on the left. However, she can now make faster progress by exploiting the skewed geometry of the terrain. That is, she can make a bigger leap forward than to the sides. In the context of gradient descent, pacing up is called acceleration. A popular class of acceleration methods is named adaptive regularization, or adaptive preconditioning, first introduced by the
AdaGrad
algorithm devised in collaboration with Prof. John Duchi from Stanford while he was at Google.
The idea is to change the geometry of the landscape of the optimization objective to make it easier for gradient descent to work. In order to do so, preconditioning methods stretch and rotate the space. The terrain after preconditioning looks like the serene, perfectly spherical lake above on the right, and the descent trajectory is a straight line! Procedurally, instead of subtracting the gradient vectors from the parameters vector per-se, adaptive preconditioning first multiplies the gradient by a 5×5 multicell structure, called a matrix preconditioner, as shown below.
This preconditioning operation yields a stretched and rotated gradient which is then subtracted as before, allowing much faster progress toward a basin. However, there is a downside to preconditioning, namely, its computational cost. Instead of subtracting a 5-dimensional gradient vector from a 5-dimensional parameter vector, the preconditioning transformation itself requires 5×5=25 operations. Suppose we would like to precondition gradients in order to learn a deep network with 10 million parameters. A single preconditioning step would require 100 trillion operations. In order to save computation, a diagonal version in which preconditioning amounts to stretching sans rotation was also introduced in the original
AdaGrad
paper. The diagonal version was later adopted and modified, yielding another very successful algorithm called
Adam
.
This simplified diagonal preconditioning imposes only a marginal additional cost to gradient descent. However, oversimplification has its own downside: we are no longer able to rotate our space. Going back to our hiker, if the deep-and-narrow canyon runs from southeast to northwest, she can no longer take large westward leaps. Had we provided her with a “rigged” compass in which the north pole is in the northwest, she could have followed her descent procedure as before. In high dimensions, the analog of compass rigging is full-matrix preconditioning. We thus asked ourselves whether we could devise a preconditioning method that is computationally efficient while allowing for the equivalent of coordinate rotations.
At Google AI Princeton, we
developed a new method
for full-matrix adaptive preconditioning at roughly the same computational cost as the commonly-used diagonal restriction. Details can be found in the paper, but the key idea behind the method is depicted below. Instead of using a full matrix, we replace the preconditioning matrix by a product of three matrices: a tall & thin matrix, a (small) square matrix, and a short & fat matrix. The vast amount of computation is performed using the smaller matrix. If we have
d
parameters, instead of a single large
d
×
d
matrix, the matrices maintained by GGT (shorthand for the operation
Gradient Gradient
T
), the proposed method, are of sizes
d
×
k
,
k
×
k
,
k
×
d
respectively.
For reasonable choices of
k
, which can be thought of as the “window size” of the algorithm, the computational bottleneck has been mitigated from a single large matrix, to that of a much smaller
kk
matrix. In our implementation we typically choose
k
to be, say, 50, and maintaining the smaller square matrix is significantly less expensive while yielding good empirical performance. When compared to other adaptive methods on standard deep learning tasks, GGT is competitive with AdaGrad and Adam.
Spectral Filtering for Control and Reinforcement Learning
Another broad mission of Google’s research group in Princeton is to develop principled building blocks for decision-making systems. In particular, the group strives to leverage provable guarantees from the field of online learning, which studies the robust (worst-case) guarantees of decision-making algorithms under uncertainty. An online algorithm is said to attain a no-regret guarantee if it learns to make decisions as well as the best "offline" decision in hindsight. Ideas from this field have already enabled many innovations within
theoretical computer science
, and provide a mathematically elegant framework to study a widely-used technique called boosting. We envision using ideas from online learning to broaden the toolkit of modern reinforcement learning.
With that goal in mind, and in collaboration with researchers and students at Princeton, we developed the algorithmic technique of
spectral filtering
for estimation and control of linear dynamical systems (see
several
recent
publications
). In this setting, noisy observations (e.g., location sensor measurements) are being streamed from an unknown source. The source of the signal is a system whose state evolves over time following a set of linear equations (e.g. Newton's laws). To forecast future signals (prediction), or to perform actions which bring the system to a desired state (control), the usual approach starts with learning the model explicitly (a task termed
system identification
), which is often slow and inaccurate.
Spectral filtering circumvents the need to model the dynamics explicitly, by reformulating prediction and control as convex programs, enabling provable no-regret guarantees. A major component of the technique is that of a new signal processing transformation. The idea is to summarize the long history of past input signals through convolution with a tailored bank of filters, and then use this representation to predict the dynamical system’s future outputs. Each filter compresses the input signal into a single real number, by taking a weighted combination of the previous inputs.
A set of filters depicted in a plot of filter amplitude versus time. With our technique of spectral filtering, multiple filters are used to predict the state of a linear dynamical system at any given time. Each filter is a set of weights used to summarize past observations, such that combining them in a weighted fashion, over time allows us to accurately predict the system.
The mathematical derivation of these weights (filters) has an interesting connection to the spectral theory of
Hankel matrices
.
Looking Forward
We are excited about the progress we have made thus far in partnership with Princeton’s faculty and students, and we look forward to the official opening of the lab in the coming weeks. It has long been Google’s view that both industry and academia benefit significantly from an open research culture, and we look forward to our continued close collaboration.
Acknowledgments
The research and results discussed in this post would not have been possible without contributions from the following researchers: Naman Agarwal, Brian Bullins, Xinyi Chen, Udaya Ghai, Tomer Koren, Karan Singh, Cyril Zhang, Yi Zhang, and visiting professor Sham Kakade. Since joining Google earlier this year, the research team has been working remotely from both the Google NYC office as well as the Princeton University campus, and they look forward to moving into the new Google space across from the Princeton campus in the weeks to come.
Labels
2018
accessibility
ACL
ACM
Acoustic Modeling
Adaptive Data Analysis
ads
adsense
adwords
Africa
AI
Algorithms
Android
Android Wear
API
App Engine
App Inventor
April Fools
Art
Audio
Augmented Reality
Australia
Automatic Speech Recognition
Awards
BigQuery
Cantonese
Chemistry
China
Chrome
Cloud Computing
Collaboration
Computational Imaging
Computational Photography
Computer Science
Computer Vision
conference
conferences
Conservation
correlate
Course Builder
crowd-sourcing
CVPR
Data Center
Data Discovery
data science
datasets
Deep Learning
DeepDream
DeepMind
distributed systems
Diversity
Earth Engine
economics
Education
Electronic Commerce and Algorithms
electronics
EMEA
EMNLP
Encryption
entities
Entity Salience
Environment
Europe
Exacycle
Expander
Faculty Institute
Faculty Summit
Flu Trends
Fusion Tables
gamification
Gboard
Gmail
Google Accelerated Science
Google Books
Google Brain
Google Cloud Platform
Google Docs
Google Drive
Google Genomics
Google Maps
Google Photos
Google Play Apps
Google Science Fair
Google Sheets
Google Translate
Google Trips
Google Voice Search
Google+
Government
grants
Graph
Graph Mining
Hardware
HCI
Health
High Dynamic Range Imaging
ICLR
ICML
ICSE
Image Annotation
Image Classification
Image Processing
Inbox
India
Information Retrieval
internationalization
Internet of Things
Interspeech
IPython
Journalism
jsm
jsm2011
K-12
KDD
Keyboard Input
Klingon
Korean
Labs
Linear Optimization
localization
Low-Light Photography
Machine Hearing
Machine Intelligence
Machine Learning
Machine Perception
Machine Translation
Magenta
MapReduce
market algorithms
Market Research
Mixed Reality
ML
ML Fairness
MOOC
Moore's Law
Multimodal Learning
NAACL
Natural Language Processing
Natural Language Understanding
Network Management
Networks
Neural Networks
NeurIPS
Nexus
Ngram
NIPS
NLP
On-device Learning
open source
operating systems
Optical Character Recognition
optimization
osdi
osdi10
patents
Peer Review
ph.d. fellowship
PhD Fellowship
PhotoScan
Physics
PiLab
Pixel
Policy
Professional Development
Proposals
Public Data Explorer
publication
Publications
Quantum AI
Quantum Computing
renewable energy
Research
Research Awards
resource optimization
Robotics
schema.org
Search
search ads
Security and Privacy
Semantic Models
Semi-supervised Learning
SIGCOMM
SIGMOD
Site Reliability Engineering
Social Networks
Software
Sound Search
Speech
Speech Recognition
statistics
Structured Data
Style Transfer
Supervised Learning
Systems
TensorBoard
TensorFlow
TPU
Translate
trends
TTS
TV
UI
University Relations
UNIX
User Experience
video
Video Analysis
Virtual Reality
Vision Research
Visiting Faculty
Visualization
VLDB
Voice Search
Wiki
wikipedia
WWW
Year in Review
YouTube
Archive
2019
Jan
2018
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2017
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2016
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Jul
May
Apr
Mar
Feb
2007
Oct
Sep
Aug
Jul
Jun
Feb
2006
Dec
Nov
Sep
Aug
Jul
Jun
Apr
Mar
Feb
Feed
Google
on
Follow @googleai
Give us feedback in our
Product Forums
.
