Google AI Blog: July 2018

Machine Learning in Google BigQuery

Wednesday, July 25, 2018

Posted by Umar Syed and Sergei Vassilvitskii, Research Scientists, Google AI, NYCGoogle BigQuerySQLBigQuery ML

gradient descent^*stochasticperform poorly when the data is suboptimally orderedfartrickierregularizationpreconditioningpaperBigQuery consoleuser guideCREATE MODEL dataset.model_name OPTIONS(model_type=’linear_reg’, input_label_cols=[‘input_label’]) AS SELECT * FROM input_table; Acknowledgements BigQuery ML is the result of a large collaboration across many teams at Google. Key contributors and sponsors include Hossein Ahmadi, Corinna Cortes, Grzegorz Czajkowski, JD Degenaar, Dan Delorey, Mingge Deng, Danielle Hanks, Amir Hormati, Abhishek Kashyap, Jing Jing Long, Dan McClary, Chris Meyers, Ross Popoff-Walker, Girishkumar Sabhnani, Vivek Sharma, Jordan Tigani, Chad Verbowski, Jiaxun Wu and Lisa Yin.
* For example, a gradient vector can be computed using the SUM and GROUP BY operators, and the weights of a model can be updated using an INNER JOIN.^↩

Announcing Cirq: An Open Source Framework for NISQ Algorithms

Wednesday, July 18, 2018

Posted by Alan Ho, Product Lead and Dave Bacon, Software Lead, Google AI Quantum Teamquantum computingquantum algorithmsNoisy Intermediate Scale Quantum^*First International Workshop on Quantum Software and Quantum Machine LearningGoogle AI Quantum teampublic alpha of Cirqcomputational problems of practical importanceApache 2

installed

OpenFermion-CirqOpenFermionlatest advances

Zapata Computing: simulation of a quantum autoencoder (example code, video tutorial)

QC Ware: QAOA implementation and integration into QC Ware’s AQUA platform (example code, video tutorial)

Quantum Benchmark: integration of True-Q software tools for assessing and extending hardware capabilities (video tutorial)

Heisenberg Quantum Simulations: simulating the Anderson Model

Cambridge Quantum Computing: integration of proprietary quantum compiler t|ket> (video tutorial)

NASA: architecture-aware compiler based on temporal-planning for QAOA (slides) and simulator of quantum computers (slides)

Bristlecone processorCirqOpenFermion-CirqAcknowledgementsWe would like to thank Craig Gidney for leading the development of Cirq, Ryan Babbush and Kevin Sung for building OpenFermion-Cirq and a whole host of code contributors to both frameworks.
* An analogous situation is how early classical programmers needed to run complex programs in very small memory spaces by paying careful attention to the lowest level details of the hardware.^↩

Improving Connectomics by an Order of Magnitude

Monday, July 16, 2018

Posted by Viren Jain, Research Scientist and Technical Lead and Michal Januszewski, Software Engineer, Connectomics at Googleconnectomicselectron microscopyneuritessynapticcolleagues at the Max Planck Institute of NeurobiologyHigh-Precision Automated Reconstruction of Neurons with Flood-Filling NetworksNature Methodsrecurrent neural networkbiorXiv3D Image Segmentation with Flood-Filling Networksimage segmentationedge detectorclassifierwatershedgraph cutrecurrent neural networksconvolutional neural network

A flood-filling network segmenting an object in 2d. The yellow dot is the center of the current area of focus; the algorithm expands the segmented region (blue) as it iteratively examines more of the overall image.

Measuring Accuracy via Expected Run Lengthmean-time-between-failurebiologically relevant quantities

Progress in expected run length (blue line) leading up to the results shared today in Nature Methods. The red line shows progress in the “merge rate,” which measures the frequency with which two separate neurites were erroneously traced as a single object; achieving a very low merge rate is important for enabling efficient strategies for manual identification and correction of the remaining errors in the reconstruction.

Songbird Connectomicszebra finchserial block-face scanning electron microscopyprevious deep learning pipelines

Our algorithm in action as it traces a single neurite in 3d in a songbird brain.

Reconstruction of a portion of zebra finch brain. Colors denote distinct objects in the segmentation that was automatically generated using a flood-filling network. Gold spheres represent synaptic locations automatically identified using a previously published approach.

test theories related to how they learn their songNext Stepsopen-sourced the TensorFlow codeWebGL visualization software for 3d datasetsAcknowledgementsWe would like to acknowledge core contributions from Tim Blakely, Peter Li, Larry Lindsey, Jeremy Maitin-Shepard, Art Pope and Mike Tyka (Google), as well as Joergen Kornfeld and Winfried Denk (Max Planck Institute).

Accelerated Training and Inference with the Tensorflow Object Detection API

Friday, July 13, 2018

Posted by Jonathan Huang, Research Scientist and Vivek Rathod, Software Engineer, Google AI Perceptionwe announcedTensorFlow Object Detection APINeural Architecture Searchinstance segmentation supportOpen Imagesfinding scofflaws on the streets of NYCdiagnosing diseases on cassava plants in Tanzania

Support for accelerated training of object detection models via Cloud TPUs

Improving the mobile deployment process by accelerating inference and making it easy to export a model to mobile with the TensorFlow Lite format

Several new model architecture definitions including:

RetinaNet (Lin et al., 2017)
A MobileNet adaptation of RetinaNet
A novel SSD-based architecture called the Pooling Pyramid Network (PPN) whose model size is >3x smaller than that of SSD MobileNet v1 with minimal loss in accuracy.

COCO datasetAccelerated Training via Cloud TPUssingle shot detectorRetinaNetmean Average Precision

Accelerated Inference via Quantization and TensorFlow Lite TensorFlow Litemodel quantizationmodel quantizationJacob et al.whitepaper by Krishnamoorthi

Quantized detection models are faster and smaller (e.g., a quantized 75% depth-reduced SSD Mobilenet model runs at >15 fps on a Pixel 2 CPU with a 4.2 Mb footprint) with minimal loss in detection accuracy compared to the full floating point model.

Try it Yourself with a New Tutorial!check out our new tutorialAcknowledgementsThis post reflects the work of the following group of core contributors: Derek Chow, Aakanksha Chowdhery, Jonathan Huang, Pengchong Jin, Zhichao Lu, Vivek Rathod, Ronny Votel and Xiangxin Zhu. We would also like to thank the following colleagues: Vasu Agrawal, Sourabh Bajaj, Chiachen Chou, Tom Jablin, Wenzhe Li, Tsung-Yi Lin, Hernan Moraldo, Kevin Murphy, Sara Robinson, Andrew Selle, Shashi Shekhar, Yash Sonthalia, Zak Stone, Pete Warden and Menglong Zhu.

Automating Drug Discoveries Using Computer Vision

Thursday, July 12, 2018

Vincent Vanhoucke, Principal Scientist, Google Brain Team“Every time you miss a protein crystal, because they are so rare, you risk missing on an important biomedical discovery.”Patrick CharbonneauMARCO initiativeProtein crystallizationMAchine Recognition of Crystallization OutcomesClassification of Crystallization Outcomes using Deep Convolutional Neural NetworksPLOS OneArXiv preprintopen-sourced our modelCloud ML Engine

Image of protein crystal, courtesy of the MARCO repository (CC-BY-4.0 license)

a large repository of curated crystallography images

Samples from the MARCO repository, illustrating the degree of variability between data sources.

Inception V3hereDuke Research Blog post

Google at ICML 2018

Monday, July 9, 2018

Posted by Christian Howard, Editor-in-Chief, Google AI CommunicationsInternational Conference on Machine LearningInternational Machine Learning SocietyTensorFlow HubMagenta projectGoogle AI ResidencICML 2018 CommitteesAndrew McCallum, Corinna Cortes, Hugo Larochelle, William CohenRyan AdamsAccepted PublicationsPredict and Constrain: Modeling Cardinality in Deep Structured PredictionNataly Brukhim, Amir GlobersonQuickshift++: Provably Good Initializations for Sample-Based Mean ShiftHeinrich Jiang, Jennifer Jang, Samory KpotufeLearning a Mixture of Two Multinomial LogitsFlavio Chierichetti, Ravi Kumar, Andrew TomkinsStructured Evolution with Compact Architectures for Scalable Policy OptimizationKrzysztof Choromanski, Mark Rowland, Vikas Sindhwani, Richard E Turner, Adrian WellerFixing a Broken ELBOAlexander Alemi, Ben Poole, Ian Fischer, Joshua Dillon, Rif Saurous, Kevin MurphyHierarchical Long-term Video Prediction without SupervisionNevan Wichers, Ruben Villegas, Dumitru Erhan, Honglak LeeSelf-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory EmbeddingsJohn Co-Reyes, Yu Xuan Liu, Abhishek Gupta, Benjamin Eysenbach, Pieter Abbeel, Sergey LevineWell Tempered LassoYuanzhi Li, Yoram SingerProgrammatically Interpretable Reinforcement LearningAbhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, Swarat ChaudhuriDynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural NetworksLechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel Schoenholz, Jeffrey PenningtonOn the Optimization of Deep Networks: Implicit Acceleration by OverparameterizationSanjeev Arora, Nadav Cohen, Elad HazanScalable Deletion-Robust Submodular Maximization: Data Summarization with Privacy and Fairness ConstraintsEhsan Kazemi, Morteza Zadimoghaddam, Amin KarbasiData Summarization at Scale: A Two-Stage Submodular ApproachMarko Mitrovic, Ehsan Kazemi, Morteza Zadimoghaddam, Amin KarbasiMachine Theory of MindNeil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, S. M. Ali Eslami, Matthew BotvinickLearning to Optimize Combinatorial FunctionsNir Rosenfeld, Eric Balkanski, Amir Globerson, Yaron SingerProportional Allocation: Simple, Distributed, and Diverse Matching with High EntropyShipra Agarwal, Morteza Zadimoghaddam, Vahab MirrokniPath Consistency Learning in Tsallis Entropy Regularized MDPsYinlam Chow, Ofir Nachum, Mohammad GhavamzadehEfficient Neural Architecture Search via Parameters SharingHieu Pham, Melody Guan, Barret Zoph, Quoc Le, Jeff DeanAdafactor: Adaptive Learning Rates with Sublinear Memory CostNoam Shazeer, Mitchell SternLearning Memory Access PatternsMilad Hashemi, Kevin Swersky, Jamie Smith, Grant Ayers, Heiner Litz, Jichuan Chang, Christos Kozyrakis, Parthasarathy RanganathanSBEED: Convergent Reinforcement Learning with Nonlinear Function ApproximationBo Dai, Albert Shaw, Lihong Li, Lin Xiao, Niao He, Zhen Liu, Jianshu Chen, Le SongScalable Bilinear Pi Learning Using State and Action FeaturesYichen Chen, Lihong Li, Mengdi WangDistributed Asynchronous Optimization with Unbounded Delays: How Slow Can You Go?Zhengyuan Zhou, Panayotis Mertikopoulos, Nicholas Bambos, Peter Glynn, Yinyu Ye, Li-Jia Li, Li Fei-FeiShampoo: Preconditioned Stochastic Tensor OptimizationVineet Gupta, Tomer Koren, Yoram SingerParallel and Streaming Algorithms for K-Core DecompositionHossein Esfandiari, Silvio Lattanzi, Vahab MirrokniCan Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games?Maithra Raghu, Alexander Irpan, Jacob Andreas, Bobby Kleinberg, Quoc Le, Jon KleinbergIs Generator Conditioning Causally Related to GAN Performance?Augustus Odena, Jacob Buckman, Catherine Olsson, Tom Brown, Christopher Olah, Colin Raffel, Ian GoodfellowThe Mirage of Action-Dependent Baselines in Reinforcement LearningGeorge Tucker, Surya Bhupatiraju, Shixiang Gu, Richard E Turner, Zoubin Ghahramani, Sergey LevineMentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted LabelsLu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, Li Fei-FeiLoss Decomposition for Fast Learning in Large Output SpacesEn-Hsu Yen, Satyen Kale, Felix Xinnan Yu, Daniel Holtmann-Rice, Sanjiv Kumar, Pradeep RavikumarA Hierarchical Latent Vector Model for Learning Long-Term Structure in MusicAdam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, Douglas EckSmoothed Action Value Functions for Learning Gaussian PoliciesOfir Nachum, Mohammad Norouzi, George Tucker, Dale SchuurmansFast Decoding in Sequence Models Using Discrete Latent VariablesLukasz Kaiser, Samy Bengio, Aurko Roy, Ashish Vaswani, Niki Parmar, Jakob Uszkoreit, Noam ShazeerAccelerating Greedy Coordinate Descent MethodsHaihao Lu, Robert Freund, Vahab MirrokniApproximate Leave-One-Out for Fast Parameter Tuning in High DimensionsShuaiwen Wang, Wenda Zhou, Haihao Lu, Arian Maleki, Vahab MirrokniImage TransformerNiki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, Dustin TranTowards End-to-End Prosody Transfer for Expressive Speech Synthesis with TacotronRJ Skerry-Ryan, Eric Battenberg, Ying Xiao, Yuxuan Wang, Daisy Stanton, Joel Shor, Ron Weiss, Robert Clark, Rif SaurousDynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural NetworksMinmin Chen, Jeffrey Pennington,, Samuel SchoenholzStyle Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech SynthesisYuxuan Wang, Daisy Stanton, Yu Zhang, RJ Skerry-Ryan, Eric Battenberg, Joel Shor, Ying Xiao, Ye Jia, Fei Ren, Rif SaurousConstrained Interacting Submodular GroupingsAndrew Cotter, Mahdi Milani Fard, Seungil You, Maya Gupta, Jeff BilmesReinforcing Adversarial Robustness using Model Confidence Induced by Adversarial TrainingXi Wu, Uyeong Jang, Jiefeng Chen, Lingjiao Chen, Somesh JhaInterpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viégas, Rory SayresOnline Learning with AbstentionCorinna Cortes, Giulia DeSalvo, Claudio Gentile, Mehryar Mohri, Scott YangOnline Linear Quadratic ControlAlon Cohen, Avinatan Hasidim, Tomer Koren, Nevena Lazic, Yishay Mansour, Kunal TalwarCompetitive Caching with Machine Learned AdviceThodoris Lykouris, Sergei VassilvitskiiEfficient Neural Audio SynthesisNal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aäron van den Oord, Sander Dieleman, Koray KavukcuogluGradient Descent with Identity Initialization Efficiently Learns Positive Definite Linear Transformations by Deep Residual NetworksPeter Bartlett, Dave Helmbold, Phil LongUnderstanding and Simplifying One-Shot Architecture SearchGabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, Quoc LeApproximation Algorithms for Cascading Prediction ModelsMatthew StreeterLearning Longer-term Dependencies in RNNs with Auxiliary LossesTrieu Trinh, Andrew Dai, Thang Luong, Quoc LeSelf-Imitation LearningJunhyuk Oh, Yijie Guo, Satinder Singh, Honglak LeeAdaptive Sampled Softmax with Kernel Based SamplingGuy Blanc, Steffen RendleWorkshops2018 Workshop on Human Interpretability in Machine Learning (WHI)Been Kim, Kush Varshney, Adrian WellerFernanda Viégas, Martin WattenbergExploration in Reinforcement LearningBen Eysenbach, Surya Bhupatiraju, Shane Gu, Junhyuk Oh, Vincent Vanhoucke, Oriol Vinyals, Doina PrecupTheoretical Foundations and Applications of Deep Generative ModelsHonglak Lee

Blog

Machine Learning in Google BigQuery

Announcing Cirq: An Open Source Framework for NISQ Algorithms

Improving Connectomics by an Order of Magnitude

Accelerated Training and Inference with the Tensorflow Object Detection API

Automating Drug Discoveries Using Computer Vision

Google at ICML 2018

Labels

Archive

Feed