Google AI Blog: April 2018

Announcing Open Images V4 and the ECCV 2018 Open Images Challenge

Monday, April 30, 2018

Posted by Vittorio Ferrari, Research Scientist, Machine PerceptionOpen ImagesupdatingrefiningOpen Images V4largest existing datasetvisualizer

Annotated images from the Open Images dataset. Left: Mark Paul Gosselaar plays the guitar by Rhys A. Right: Civilization by Paul Downey. Both images used under CC BY 2.0 license.

Open Images Challenge2018 European Conference on Computer VisionPASCAL VOCImageNetCOCO

12.2M bounding-box annotations for 500 categories on 1.7M training images,

A broader range of categories than previous detection challenges, including new objects such as “fedora” and “snowman”.

In addition to the object detection main track, the challenge includes a Visual Relationship Detection track, on detecting pairs of objects in particular relations, e.g. “woman playing guitar”.

crowdsource.google.com

Google at ICLR 2018

Sunday, April 29, 2018

Posted by Jeff Dean, Google Senior Fellow, Head of Google Research and Machine Intelligence6th International Conference on Learning Representationsmachine learningneural networksdeep learningblueSenior Program Chair:Tara SainathSteering Committee includes:Hugo LarochelleOral ContributionsWasserstein Auto-EncodersIlya Tolstikhin, Olivier Bousquet, Sylvain Gelly, Bernhard ScholkopfOn the Convergence of Adam and Beyond(Best Paper Award)Sashank J. Reddi, Satyen Kale, Sanjiv KumarAsk the Right Questions: Active Question Reformulation with Reinforcement LearningChristian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, Wei WangBeyond Word Importance: Contextual Decompositions to Extract Interactions from LSTMsW. James Murdoch, Peter J. Liu, Bin YuConference PostersBoosting the Actor with Dual CriticBo Dai, Albert Shaw, Niao He, Lihong Li, Le SongMaskGAN: Better Text Generation via Filling in the _______William Fedus, Ian Goodfellow, Andrew M. DaiScalable Private Learning with PATENicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Ulfar ErlingssonDeep Gradient Compression: Reducing the Communication Bandwidth for Distributed TrainingYujun Lin, Song Han, Huizi Mao, Yu Wang, William J. DallyFlipout: Efficient Pseudo-Independent Weight Perturbations on Mini-BatchesYeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, Roger GrosseLatent Constraints: Learning to Generate Conditionally from Unconditional Generative ModelsAdam Roberts, Jesse Engel, Matt HoffmanMulti-Mention Learning for Reading Comprehension with Neural CascadesSwabha Swayamdipta, Ankur P. Parikh, Tom KwiatkowskiQANet: Combining Local Convolution with Global Self-Attention for Reading ComprehensionAdams Wei Yu, David Dohan, Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V. LeSensitivity and Generalization in Neural Networks: An Empirical StudyRoman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-DicksteinAction-dependent Control Variates for Policy Optimization via Stein IdentityHao Liu, Yihao Feng, Yi Mao, Dengyong Zhou, Jian Peng, Qiang LiuAn Efficient Framework for Learning Sentence RepresentationsLajanugen Logeswaran, Honglak Lee Fidelity-Weighted LearningMostafa Dehghani, Arash Mehrjou, Stephan Gouws, Jaap Kamps, Bernhard Schölkopf Generating Wikipedia by Summarizing Long SequencesPeter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, Noam ShazeerMatrix Capsules with EM RoutingGeoffrey Hinton, Sara Sabour, Nicholas Frosst Temporal Difference Models: Model-Free Deep RL for Model-Based ControlSergey Levine, Shixiang Gu, Murtaza Dalal, Vitchyr Pong Deep Neural Networks as Gaussian ProcessesJaehoon Lee, Yasaman Bahri, Roman Novak, Samuel L. Schoenholz, Jeffrey Pennington, Jascha Sohl-DicksteinMany Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence at Every StepWilliam Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M. Dai, Shakir Mohamed, Ian Goodfellow Initialization Matters: Orthogonal Predictive State Recurrent Neural NetworksKrzysztof Choromanski, Carlton Downey, Byron BootsLearning Differentially Private Recurrent Language ModelsH. Brendan McMahan, Daniel Ramage, Kunal Talwar, Li ZhangLearning Latent Permutations with Gumbel-Sinkhorn NetworksGonzalo Mena, David Belanger, Scott Linderman, Jasper Snoek Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement LearningBenjamin Eysenbach, Shixiang Gu, Julian Ibarz, Sergey LevineMeta-Learning for Semi-Supervised Few-Shot ClassificationMengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Josh Tenenbaum, Hugo Larochelle, Richard ZemelThermometer Encoding: One Hot Way to Resist Adversarial ExamplesJacob Buckman, Aurko Roy, Colin Raffel, Ian GoodfellowA Hierarchical Model for Device PlacementAzalia Mirhoseini, Anna Goldie, Hieu Pham, Benoit Steiner, Quoc V. Le, Jeff Dean Monotonic Chunkwise AttentionChung-Cheng Chiu, Colin RaffelTraining Confidence-calibrated Classifiers for Detecting Out-of-Distribution SamplesKimin Lee, Honglak Lee, Kibok Lee, Jinwoo ShinTrust-PCL: An Off-Policy Trust Region Method for Continuous ControlOfir Nachum, Mohammad Norouzi, Kelvin Xu, Dale SchuurmansEnsemble Adversarial Training: Attacks and DefensesFlorian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel Stochastic Variational Video PredictionMohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy Campbell, Sergey LevineDepthwise Separable Convolutions for Neural Machine TranslationLukasz Kaiser, Aidan N. Gomez, Francois CholletDon’t Decay the Learning Rate, Increase the Batch Size Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, Quoc V. LeGenerative Models of Visually Grounded ImaginationRamakrishna Vedantam, Ian Fischer, Jonathan Huang, Kevin MurphyLarge Scale Distributed Neural Network Training through Online DistillationRohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, Geoffrey E. HintonLearning a Neural Response Metric for Retinal ProsthesisNishal P. Shah, Sasidhar Madugula, Alan Litke, Alexander Sher, EJ Chichilnisky, Yoram Singer, Jonathon ShlensNeumann Optimizer: A Practical Optimization Algorithm for Deep Neural NetworksShankar Krishnan, Ying Xiao, Rif A. SaurousA Neural Representation of Sketch DrawingsDavid Ha, Douglas EckDeep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson SamplingCarlos Riquelme, George Tucker, Jasper SnoekGeneralizing Hamiltonian Monte Carlo with Neural NetworksDaniel Levy, Matthew D. Hoffman, Jascha Sohl-DicksteinLeveraging Grammar and Reinforcement Learning for Neural Program SynthesisRudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, Pushmeet Kohli On the Discrimination-Generalization Tradeoff in GANsPengchuan Zhang, Qiang Liu, Dengyong Zhou, Tao Xu, Xiaodong HeA Bayesian Perspective on Generalization and Stochastic Gradient DescentSamuel L. Smith, Quoc V. Le Learning how to Explain Neural Networks: PatternNet and PatternAttributionPieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, Sven DähneSkip RNN: Learning to Skip State Updates in Recurrent Neural NetworksVíctor Campos, Brendan Jou, Xavier Giró-i-Nieto, Jordi Torres, Shih-Fu ChangTowards Neural Phrase-based Machine TranslationPo-Sen Huang, Chong Wang, Sitao Huang, Dengyong Zhou, Li DengUnsupervised Cipher Cracking Using Discrete GANsAidan N. Gomez, Sicong Huang, Ivan Zhang, Bryan M. Li, Muhammad Osama, Lukasz Kaiser Variational Image Compression With A Scale HyperpriorJohannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, Nick JohnstonWorkshop PostersLocal Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter ValuesJulius Adebayo, Justin Gilmer, Ian Goodfellow, Been KimStoachastic Gradient Langevin Dynamics that Exploit Neural Network StructureZachary Nado, Jasper Snoek, Bowen Xu, Roger Grosse, David Duvenaud, James MartensTowards Mixed-initiative generation of multi-channel sequential structureAnna Huang, Sherol Chen, Mark J. Nelson, Douglas EckCan Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games?Maithra Raghu, Alex Irpan, Jacob Andreas, Robert Kleinberg, Quoc V. Le, Jon KleinbergGILBO: One Metric to Measure Them AllAlexander Alemi, Ian FischerHoME: a Household Multimodal EnvironmentSimon Brodeur, Ethan Perez, Ankesh Anand, Florian Golemo, Luca Celotti, Florian Strub, Jean Rouat, Hugo Larochelle, Aaron CourvilleLearning to Learn without LabelsLuke Metz, Niru Maheswaranathan, Brian Cheung, Jascha Sohl-DicksteinLearning via Social Awareness: Improving Sketch Representations with Facial FeedbackNatasha Jaques, Jesse Engel, David Ha, Fred Bertsch, Rosalind Picard, Douglas EckNegative Eigenvalues of the Hessian in Deep Neural Networks Guillaume Alain, Nicolas Le Roux, Pierre-Antoine ManzagolRealistic Evaluation of Semi-Supervised Learning AlgorithmsAvital Oliver, Augustus Odena, Colin Raffel, Ekin Cubuk, lan GoodfellowWinner's Curse? On Pace, Progress, and Empirical Rigor D. Sculley, Jasper Snoek, Alex Wiltschko, Ali RahimiMeta-Learning for Batch Mode Active LearningSachin Ravi, Hugo Larochelle To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model CompressionMichael Zhu, Suyog Gupta Adversarial SpheresJustin Gilmer, Luke Metz, Fartash Faghri, Sam Schoenholz, Maithra Raghu,,Martin Wattenberg, Ian GoodfellowClustering Meets Implicit Generative ModelsFrancesco Locatello, Damien Vincent, Ilya Tolstikhin, Gunnar Ratsch, Sylvain Gelly, Bernhard ScholkopfDecoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity TasksVitalii Zhelezniak, Dan Busbridge, April Shen, Samuel L. Smith, Nils Y. HammerlaLearning Longer-term Dependencies in RNNs with Auxiliary LossesTrieu Trinh, Quoc Le, Andrew Dai, Thang LuongGraph Partition Neural Networks for Semi-Supervised ClassificationAlexander Gaunt, Danny Tarlow, Marc Brockschmidt, Raquel Urtasun, Renjie Liao, Richard ZemelSearching for Activation FunctionsPrajit Ramachandran, Barret Zoph, Quoc LeTime-Dependent Representation for Neural Event Sequence PredictionYang Li, Nan Du, Samy BengioFaster Discovery of Neural Architectures by Searching for Paths in a Large ModelHieu Pham, Melody Guan, Barret Zoph, Quoc V. Le, Jeff DeanIntriguing Properties of Adversarial ExamplesEkin Dogus Cubuk, Barret Zoph, Sam Schoenholz, Quoc Le PPP-Net: Platform-aware Progressive Search for Pareto-optimal Neural ArchitecturesJin-Dong Dong, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, Min SunThe Mirage of Action-Dependent Baselines in Reinforcement LearningGeorge Tucker, Surya Bhupatiraju, Shixiang Gu, Richard E. Turner, Zoubin Ghahramani, Sergey LevineLearning to Organize Knowledge with N-Gram MachinesFan Yang, Jiazhong Nie, William W. Cohen, Ni LaoOnline variance-reducing optimizationNicolas Le Roux, Reza Babanezhad, Pierre-Antoine Manzagol

Announcing the Google Cloud Platform Research Credits Program

Thursday, April 26, 2018

Posted by Steven Butschi, Head of Higher Education, Googlemachine learning tools to understand and address student questionsstudy robotic interactions with humansCloud Platform (GCP) research creditsmade the transitionopen-source platform to manage the complex data sets of climate sciencevariant analysis pipelineeligiblewebsiteapply now

Google’s Workshop on AI/ML Research and Practice in India

Tuesday, April 24, 2018

Posted by Pankaj Gupta and Anand Rangarajan, Engineering Directors, Google IndiaWorkshop on Artificial Intelligence and Machine LearninghereJeff DeanPrabhakar Raghavandeep learning to solve challenging problemsreinventing productivity using AIRajen ShethRoberto Bayardopanel discussionPankaj GuptaAshish Tendulkar

Prabhakar Raghavan’s keynote address

Sessionsagenda

Deep Learning: Balaraman Ravindran (IIT Madras) opened the session discussing the structure in deep reinforcement learning, then Sunita Sarawagi (IIT Bombay) discussed domain generalization via cross-gradient training. Tanuja Ganu (DataGlen Technologies) spoke about creating intelligent and sustainable energy systems and finally Rushi Bhatt (LinkedIn) spoke about maintaining professional quality of the LinkedIn feed using deep learning.

Natural Language Processing: Sourangshu Bhattacharya (IIT Kharagpur) opened the session talking about task specific representation learning for web-scale entity disambiguation followed by Mausam (IIT Delhi) who delved into hierarchical pointer memory network for task-oriented dialogue. Partha Talukdar (IISc) presented his work on canonicalizing open knowledge graphs then Manish Gupta (VideoKen) discussed his findings in applying machine learning to support human learning. Finally, Arjun Maheshwaran (Agara Labs) and Pankaj Gupta (Google Bangalore) presented their joint-research and industry work on smart customer support.

Machine Learning Systems: Goda Ramkumar (Ola) opened the session discussing her findings about ML/data science algorithms that power Ola. Prateek Jain (Microsoft Research India) presented his research on resource-efficient ML in 2 KB RAM for the Internet of Things. Animesh Mukherjee (IIT Kharagpur) discussed his yet-to-published work on ranking state-of-the-art papers via incomplete tournaments induced by citations from performance tables. Finally, Srujana Merugu (Independent ML Consultant) provided a practitioner's perspective on building ML Systems.

Computer Vision: Vineeth Balsubramanian (IIT Hyderabad) opened the session discussing his research on generalized gradient-based visual explanations for deep convolutional networks (Grad-CAM++). Vinay Namboodiri (IIT Kanpur) spoke about his research on adversarial machine learning. Vishnu Makkapati (Myntra) discussed his research and practice in generation of fashion designs using generative adversarial networks. Varun Gulshan (Google Brain) spoke about his research on using ML in medical imaging. Finally, Soma Biswas (IISc) presented her research on preserving semantic similarity for zero-shot learning.

Reinforcement Learning, Generative Models and more: Aditya Gopalan (IISc) opened the session describing online and reinforcement learning in complex environments. Then Ganesh Ramakrishnan (IIT Bombay) presented research on human assisted machine learning. Lavanya Tekumalla (Amazon) presented her work on using Bayesian models for product size recommendation. Kabir Rustogi (Delhivery) discussed his work on addressing geocoding using graph based machine learning models. Mohit Kumar (Flipkart) presented his research and practice on fraudulent user prediction in ratings platform. The workshop ended with a talk by Rahul Sami (Google) who discussed ML challenges in Google Maps auto-moderation.

Pankaj Gupta moderating the panel discussion

Summary and Next Stepsresearch awards round

Introducing the CVPR 2018 On-Device Visual Intelligence Challenge

Friday, April 20, 2018

Posted by Bo Chen, Software Engineer and Jeffrey M. Gilbert, Member of Technical Staff, Google ResearchMobileNet model familyinteger quantizationOn-device Visual Intelligence ChallengeLow-Power Image Recognition Challenge WorkshopCVPR2018A Unified Metric¹multiply-accumulate operationsdeep neural network

Even though the number of multiply-accumulate operations (# MACs) is the most commonly used metric to approximate on-device latency, it is a poor predictor of latency. Using data from various quantized and floating point MobileNet V1 and V2 based models, this graph plots on-device latency on a common reference device versus the number of MACs. It is clear that models with similar latency can have very different MACs, and vice versa.

An Evaluation Platformoptimized kernels for mobile CPUslight-weight portable model formats

TOCO compiler for optimizing TensorFlow models for efficient inference

TensorFlow Lite inference engine for mobile deployment

A benchmarking SDK that can be run locally on any Android phone

Sample models to showcase successful mobile architectures that run inference in floating-point and quantized modes

Google’s benchmarking tool for reliable latency measurements on specific Pixel phones (available to registered contestants).

tutorial

Sampling of current MobileNet mobile models illustrating the tradeoff between increased accuracy and reduced latency.

hereAcknowledgementsWe would like to acknowledge our core contributors Achille Brighton, Alec Go, Andrew Howard, Hartwig Adam, Mark Sandler and Xiao Zhang. We would also like to acknowledge our external collaborators Alex Berg and Yung-Hsiang Lu. We give special thanks to Andre Hentz, Andrew Selle, Benoit Jacob, Brad Krueger, Dmitry Kalenichenko, Megan Cummins, Pete Warden, Rajat Monga, Shiyu Hu and Yicheng Fan.
1 Alternatively the same metric could encourage even lower power operation by only processing a subset of the images in the input stream.^↩

DeepVariant Accuracy Improvements for Genetic Datatypes

Thursday, April 19, 2018

Posted by Pi-Chuan Chang, Software Engineer and Lizzie Dorfman, Technical Program Manager, Google Brain Teamreleased DeepVariantgreater accuracy than previous methodsDeepVariant v0.6whole exome sequencingpolymerase chain reactionMany Types of Sequencing DataTruth Sets for Variant CallingDeep learningGenome in a BottleNational Institute of Standards and Technologyfirst benchmarking genome released by GIABInternational HapMap ProjectImproved Exome Model in v0.5v0.5 releasecodes for proteinsDNAnexustraining dataindelsingle nucleotide polymorphism

The total number of exome errors for HG002 across DeepVariant versions, broken down by indel errors (left) and SNP errors (right). Errors are either false positive (FP), colored yellow, or false negative (FN), colored blue. The largest accuracy jump is between v0.4 and v0.5, largely attributable to a reduction in indel FPs.

Improved Whole Genome Sequencing Model for PCR+ data in v0.6v0.6polymerase chain reaction

DeepVariant v0.6 shows major accuracy improvements for PCR+ data, largely attributable to a reduction in indel errors. Here we re-analyze two PCR+ samples that were used in external evaluations, including DNAnexus on the left (see details in figure 10) and bcbio on the right, showing how indel accuracy improves with each DeepVariant version.

DNAnexusbcbioLooking Forwardlet us know

An Augmented Reality Microscope for Cancer Detection

Monday, April 16, 2018

Posted by Martin Stumpe, Technical Lead and Craig Mermel, Product Manager, Google Brain Teamdeep learningophthalmologydermatologyradiologypathologypublished resultsconvolutional neural networktalkAACRAn Augmented Reality Microscope for Real-time Automated Detection of Cancer TensorFlow

Left: Schematic overview of the ARM. A digital camera captures the same field of view (FoV) as the user and passes the image to an attached compute unit capable of running real-time inference of a machine learning model. The results are fed back into a custom AR display which is inline with the ocular lens and projects the model output on the same plane as the slide. Right: A picture of our prototype which has been retrofitted into a typical clinical-grade light microscope.

Example view through the lens of the ARM. These images show examples of the lymph node metastasis model with 4x, 10x, 20x, and 40x microscope objectives.

Introducing Semantic Experiences with Talk to Books and Semantris

Friday, April 13, 2018

Posted by Ray Kurzweil, Director of Engineering and Rachel Bernstein, Product Manager, Google ResearchNatural language understandingword vectorsmake improvements to Smart Reply for GmailSemantic ExperiencesTalk to BooksSemantrisUniversal Sentence Encoderpretrained semantic TensorFlow moduleModeling approachEfficient Natural Language Response for Smart ReplyTalk to BooksTalk to Booksto

Talk to Books

SemantrisSemantrisTry it out yourself

Semantris Arcade

TensorFlow modelsAcknowledgementsTalk to Books was developed by Aaron Phillips, Amin Ahmad, Rachel Bernstein, Aaron Cohen, Noah Constant, Ray Kurzweil, Igor Krivokon, Vladimir Magay, Peter McKenzie, Bryan Richter, Chris Tar, Dave Uthus, and Ni Yan. Semantris was developed by Ben Pietrzak, RJ Mical, Steve Pucci, Maria Voitovich, Mo Adeleye, Diana Huang, Catherine McCurry, Tomomi Sohn, and Connor Moore. Core semantic search technology development was led by Brian Strope and Yunhsuan Sung. Fast scalable matching work was led by Sanjiv Kumar, Dave Dopson, and David Simcha. We'd also like to acknowledge Hallie Benjamin, Eric Breck, Mario Guajardo-Céspedes, Yoni Halpern, Margaret Mitchell, Ben Packer, Andrew Smart and Lucy Vasserman.

Seeing More with In Silico Labeling of Microscopy Images

Thursday, April 12, 2018

Eric Christiansen, Senior Software Engineer, Google ResearchmicroscopyTransmitted lightFluorescence microscopyautomatically assess the quality of imagesassist pathologists diagnosing cancerous tissueIn Silico Labeling: Predicting Fluorescent Labels in Unlabeled ImagesCellopen sourced our networkBackgroundphase-contrastphase-shifted

Transmitted light (phase-contrast) image of a human motor neuron culture derived from induced pluripotent stem cells. Outset 1 shows a cluster of cells, possibly neurons. Outset 2 shows a flaw in the image obscuring underlying cells. Outset 3 shows neurites. Outset 4 shows what appear to be dead cells. Scale bar is 40 μm. Source images for this and the following figures come from the Finkbeiner lab at the Gladstone Institutes.

x, yz

A phase-contrast z-stack of the same cells. Note how the appearance changes as the focus is shifted. Now we can see that the fuzzy shape in the lower right of Outset 1 is a single oblong cell, and that the rightmost cell in Outset 4 is taller than the uppermost cell, possibly indicating that it has undergone programmed cell death.

Fluorescence microscopy image of the same cells. The blue fluorescent label localizes to DNA, highlighting cell nuclei. The green fluorescent label localizes to a protein found only in dendrites, a neural substructure. The red fluorescent label localizes to a protein found only in axons, another neural substructure. With these labels it is much easier to understand what’s happening in the sample. For example, the green and red labels in Outset 1 confirm this is a neural cluster. The red label in Outset 3 shows that the neurites are axons, not dendrites. The upper-left blue dot in Outset 4 reveals a previously hard-to-see nucleus, and the lack of a blue dot for the cell at the left shows it to be DNA-free cellular debris.

Seeing more with deep learning

Overview of our system. (A) The dataset of training examples: pairs of transmitted light images from z-stacks with pixel-registered sets of fluorescence images of the same scene. Several different fluorescent labels were used to generate fluorescence images and were varied between training examples; the checkerboard images indicate fluorescent labels which were not acquired for a given example. (B) The untrained deep network was (C) trained on the data A. (D) A z-stack of images of a novel scene. (E) The trained network, C, is used to predict fluorescence labels learned from A for each pixel in the novel images, D.

Inceptionin-scaledown-scaleup-scaleGoogle HypertuneSteve Finkbeiner's lab at the Gladstone InstitutesRubin Lab at Harvardbright-fieldphase-contrastdifferential interference contrastmotor neuronsinduced pluripotent stem cellscortical cultures

Animation showing the same cells in transmitted light and fluorescence imaging, along with predicted fluorescence labels from our model. Outset 2 shows the model predicts the correct labels despite the artifact in the input image. Outset 3 shows the model infers these processes are axons, possibly because of their distance from the nearest cells. Outset 4 shows the model sees the hard-to-see cell at the top, and correctly identifies the object at the left as DNA-free cell debris.

Try it for yourself!open sourced our modelsingle imagetransfer learningcheck out the codeAcknowledgementsWe thank the Google Accelerated Science team for originating and developing this project and its publication, and additionally Kevin P. Murphy for supporting its publication. We thank Mike Ando, Youness Bennani, Amy Chung-Yu Chou, Jason Freidenfelds, Jason Miller, Kevin P. Murphy, Philip Nelson, Patrick Riley, and Samuel Yang for ideas and editing help with this post. This study was supported by NINDS (NS091046, NS083390, NS101995), the NIH’s National Institute on Aging (AG065151, AG058476), the NIH’s National Human Genome Research Institute (HG008105), Google, the ALS Association, and the Michael J. Fox Foundation.

Looking to Listen: Audio-Visual Speech Separation

Wednesday, April 11, 2018

Posted by Inbar Mosseri and Oran Lang, Software Engineers, Google Researchcocktail party effectLooking to Listen at the Cocktail PartySIGGRAPH 2018

The input to our method is a video with one or more people speaking, where the speech of interest is interfered by other speakers and/or background noise. The output is a decomposition of the input audio track into clean speech tracks, one for each person detected in the video.

An Audio-Visual Speech Separation ModelAudioSetspectrogramour paper

Our multi-stream, neural network-based model architecture.

Application to Speech Recognitionproject web pagethis work from UC Berkeleythis work from MITAcknowledgementsThe research described in this post was done by Ariel Ephrat (as an intern), Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, Bill Freeman and Michael Rubinstein. We would like to thank Yossi Matias and Google Research Israel for their support for the project, and John Hershey for his valuable feedback. We also thank Arkady Ziefman for his help with animations and figures, and Rachel Soh for helping us procure permissions for video content in our results.

Blog

Labels

Archive

Feed