Google AI Blog: 2018

Advances in Semantic Textual Similarity

Thursday, May 17, 2018

Posted by Yinfei Yang, Software Engineer and Chris Tar, Engineering Manager, Google AISmart ComposeTalk to BooksTensorFlow HubSemantic Textual SimilarityLearning Semantic Textual Similarity from Conversations

Sentences are semantically similar if they can be answered by the same responses. Otherwise, they are semantically different.

SNLIentailmentSTSBenchmarkCQA task B

For a given input, classification is considered a ranking problem against potential candidates.

Universal Sentence EncoderUniversal Sentence Encoderskip-thought

Pairwise semantic similarity comparison via outputs from TensorFlow Hub Universal Sentence Encoder.

deep average networkTransformer

Multi-task training as described in “Universal Sentence Encoder”. A variety of tasks and task structures are joined by shared encoder layers/parameters (grey boxes).

New Models modelnewTensorFlow HubUniversal Sentence Encoder - LargeUniversal Sentence Encoder - Lite

The Large model is trained with the Transformer encoder described in our second paper. It targets scenarios requiring high precision semantic representations and the best model performance at the cost of speed & size.

The Lite model is trained on a Sentence Piece vocabulary instead of words in order to significantly reduce the vocabulary size, which is a major contributor of model size. It targets scenarios where resources like memory and CPU are limited, such as on-device or browser based implementations.

AcknowledgementsDaniel Cer, Mario Guajardo-Cespedes, Sheng-Yi Kong, Noah Constant for training the models, Nan Hua, Nicole Limtiaco, Rhomni St. John for transferring tasks, Steve Yuan, Yunhsuan Sung, Brian Strope, Ray Kurzweil for discussion of the model architecture. Special thanks to Sheng-Yi Kong and Noah Constant for training the Lite model.

Smart Compose: Using Neural Networks to Help Write Emails

Wednesday, May 16, 2018

Posted by Yonghui Wu, Principal Engineer, Google Brain TeamGoogle I/OSmart ComposeSmart Reply

Latency: Since Smart Compose provides predictions on a per-keystroke basis, it must respond ideally within 100ms for the user not to notice any delays. Balancing model complexity and inference speed was a critical issue.

Scale: Gmail is used by more than 1.4 billion diverse users. In order to provide auto completions that are useful for all Gmail users, the model has to have enough modeling capacity so that it is able to make tailored suggestions in subtly different contexts.

Fairness and Privacy: In developing Smart Compose, we needed to address sources of potential bias in the training process, and had to adhere to the same rigorous user privacy standards as Smart Reply, making sure that our models never expose user’s private information. Furthermore, researchers had no access to emails, which meant they had to develop and train a machine learning system to work on a dataset that they themselves cannot read.

Finding the Right Model ngramneural bag-of-wordsRNN languagesequence-to-sequenceword embeddings

Smart Compose RNN-LM model architecture. Subject and previous email message are encoded by averaging the word embeddings in each field. The averaged embeddings are then fed to the RNN-LM at each decoding step.

Accelerated Model Training & ServingTPUv2 PodFairness and PrivacyFairness in machine learningSemantics derived automatically from language corpora contain human-like biasesthis paperFuture workTransformer,RNMT+AcknowledgementsSmart Compose language generation model was developed by Benjamin Lee, Mia Chen, Gagan Bansal, Justin Lu, Jackie Tsay, Kaushik Roy, Tobias Bosch, Yinan Wang, Matthew Dierker, Katherine Evans, Thomas Jablin, Dehao Chen, Vinu Rajashekhar, Akshay Agrawal, Yuan Cao, Shuyuan Zhang, Xiaobing Liu, Noam Shazeer, Andrew Dai, Zhifeng Chen, Rami Al-Rfou, DK Choe, Yunhsuan Sung, Brian Strope, Timothy Sohn, Yonghui Wu, and many others.

Automatic Photography with Google Clips

Friday, May 11, 2018

Posted by Aseem Agarwala, Research Scientist, Clips Content Team LeadTo me, photography is the simultaneous recognition, in a fraction of a second, of the significance of an event as well as of a precise organization of forms which give that event its proper expression.Henri Cartier-BressonGoogle Clips

We wanted all computations to be performed on-device. In addition to extending battery life and reducing latency, on-device processing means that none of your clips leave the device unless you decide to save or share them, which is a key privacy control.

We wanted the device to capture short videos, rather than single photographs. Moments with motion can be more poignant and true-to-memory, and it is often easier to shoot a video around a compelling moment than it is to capture a perfect, single instant in time.

We wanted to focus on capturing candid moments of people and pets, rather than the more abstract and subjective problem of capturing artistic images. That is, we did not attempt to teach Clips to think about composition, color balance, light, etc.; instead, Clips focuses on selecting ranges of time containing people and animals doing interesting activities.

Learning to Recognize Great MomentsWe then hired

Training a Clips Quality Modelinrecognize over 27,000 different labelsMobileNetpiecewise linear regression model

Diagram of the training process for generating frame quality scores. Piecewise linear regression maps from an ICM embedding to a score which, when averaged across a video segment, yields a moment score. The moment score of the preferred segment should be higher.

most recent releaseShot Control

Respect Power & Thermals: We want the Clips battery to last roughly three hours, and we don’t want the device to overheat — the device can’t run at full throttle all the time. Clips spends much of its time in a low-power mode that captures one frame per second. If the quality of that frame exceeds a threshold set by how much Clips has recently shot, it moves into a high-power mode, capturing at 15 fps. Clips then saves a clip at the first quality peak encountered.

Avoid Redundancy: We don’t want Clips to capture all of its moments at once, and ignore the rest of a session. Our algorithms therefore cluster moments into visually similar groups, and limit the number of clips in each cluster.

The Benefit of Hindsight: It’s much easier to determine which clips are the best when you can examine the totality of clips captured. Clips therefore captures more moments than it intends to show to the user. When clips are ready to be transferred to the phone, the Clips device takes a second look at what it has shot, and only transfers the best and least redundant content.

Machine Learning Fairnessfairness in ML algorithmsConclusionAcknowledgementsThe algorithms described here were conceived and implemented by a large group of Google engineers, research scientists, and others. Figures were made by Lior Shapira. Thanks to Lior and Juston Payne for video content.

Custom On-Device ML Models with Learn2Compress

Wednesday, May 9, 2018

Posted by Sujith Ravi, Senior Staff Research Scientist, Google Expander TeamOn-device machine learningMobileNetsProjectionNetsownML KitTensorFlow LitehereHow it WorksProjectionNet

Learn2Compress for automatically generating on-device ML models.

Pruning reduces model size by removing weights or operations that are least useful for predictions (e.g.low-scoring weights). This can be very effective especially for on-device models involving sparse inputs or outputs, which can be reduced up to 2x in size while retaining 97% of the original prediction quality.

Quantization techniques are particularly effective when applied during training and can improve inference speed by reducing the number of bits used for model weights and activations. For example, using 8-bit fixed point representation instead of floats can speed up the model inference, reduce power and further reduce size by 4x.

Joint training and distillation approaches follow a teacher-student learning strategy — we use a larger teacher network (in this case, user-provided TensorFlow model) to train a compact student network (on-device model) with minimal loss in accuracy.

Joint training and distillation approach to learn compact student models.

The teacher network can be fixed (as in distillation) or jointly optimized, and even train multiple student models of different sizes simultaneously. So instead of a single model, Learn2Compress generates multiple on-device models in a single shot, at different sizes and inference speeds, and lets the developer pick one best suited for their application needs.

transfer learningHow well does it work?MobileNetsNASNetInceptionProjectionNet

Accuracy at various sizes for Learn2Compress models and full-sized baseline networks on CIFAR-10 (left) and ImageNet (right) image classification tasks. Student networks used to produce the compressed variants for CIFAR-10 and ImageNet are modeled using NASNet and MobileNet-inspired architectures, respectively.

ImageNetCIFAR-10

Computation cost and average prediction latency (on Pixel phone) for baseline and Learn2Compress models on CIFAR-10 image classification task. Learn2Compress-optimized models use NASNet-style network architecture.

FishbrainAcknowledgmentsI would like to acknowledge our core contributors Gaurav Menghani, Prabhu Kaliamoorthi and Yicheng Fan along with Wei Chai, Kang Lee, Sheng Xu and Pannag Sanketi. Special thanks to Dave Burke, Brahim Elbouchikhi, Hrishikesh Aradhye, Hugues Vincent, and Arun Venkatesan from the Android team; Sachin Kotwani, Wesley Tarle, Pavel Jbanov and from the Firebase team; Andrei Broder, Andrew Tomkins, Robin Dua, Patrick McGregor, Gaurav Nemade, the Google Expander team and TensorFlow team.

Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone

Tuesday, May 8, 2018

Posted by Yaniv Leviathan, Principal Engineer and Yossi Matias, Vice President, Engineering, GoogleGoogle voice searchWaveNet

Duplex scheduling a hair salon appointment:

Duplex calling a restaurant:

Conducting Natural Conversations“So umm Tuesday through Thursday we are open 11 to 2, and then reopen 4 to 9, and then Friday, Saturday, Sunday we... or Friday, Saturday we're open 11 to 9 and then Sunday we're open 1 to 9.”

Example of complex statement:

elaborations syncsinterruptionspausesEnter Duplexunderstandinginteractingtimingspeakingrecurrent neural networkTensorFlow Extended

Incoming sound is processed through an ASR system. This produces text that is analyzed with context data and other inputs to produce a response text that is read aloud through the TTS system.

Duplex handling interruptions:

Duplex elaborating:

Duplex responding to a sync:

Sounding NaturalTacotronWaveNetlatencymoreSystem Operationfully autonomouslyreal-time supervised trainingBenefits for Businesses and Users

Duplex calling a restaurant:

Duplex asking for holiday hours:

A user asks the Google Assistant for an appointment, which the Assistant then schedules by having Duplex call the business.

Google Assistant

Yaniv Leviathan, Google Duplex lead, and Matan Kalman, engineering manager on the project, enjoying a meal booked through a call from Duplex.

Duplex calling to book the above meal:

Deep Learning for Electronic Health Records

Tuesday, May 8, 2018

Posted by Alvin Rajkomar MD, Research Scientist and Eyal Oren PhD, Product Manager, Google AI

Scalable: Predictions should be straightforward to create for any important outcome and for different hospital systems. Since healthcare data is very complicated and requires much data wrangling, this requirement is not straightforward to satisfy.

Accurate: Predictions should alert clinicians to problems but not distract them with false alarms. With the widespread adoption of electronic health records, we set out to use that data to create more accurate prediction models.

UC San FranciscoStanford MedicineThe University of Chicago MedicineScalable and Accurate Deep Learning with Electronic Health RecordsNature Partner Journals: Digital Medicine

ScalabilityFast Healthcare Interoperability Resourcesin an earlier blog postrecurrent neural networksfeedforward

Data in a patient's record is represented as a timeline. For illustrative purposes, we display various types of clinical data (e.g. encounters, lab tests) by row. Each piece of data, indicated as a little grey dot, is stored in FHIR, an open data standard that can be used by any healthcare institution. A deep learning model analyzed a patient's chart by reading the timeline from left to right, from the beginning of a chart to the current hospitalization, and used this data to make different types of predictions.

Prediction Accuracyarea-under-the-receiver-operator curvenot

A deep learning model was used to render a prediction 24 hours after a patient was admitted to the hospital. The timeline (top of figure) contains months of historical data and the most recent data is shown enlarged in the middle. The model "attended" to information highlighted in red that was in the patient's chart to "explain" its prediction. In this case-study, the model highlighted pieces of information that make sense clinically. Figure from our paper.

What does this mean for patients and clinicians?

Introducing Google AI

Monday, May 7, 2018

Posted by Christian Howard, Editor-in-Chief, Google AI CommunicationsHmm, have I made a wrong turn? I was looking for Google Research…computer visionhealthcare researchAutoMLproductsplatforms

Google AI website

The Question of Quantum Supremacy

Friday, May 4, 2018

Posted by Sergio Boixo, Research Scientist and Theory Team Lead, and Charles Neill, Quantum Electronics Engineer, Quantum A.I. LabQuantum computinginformation technologyquantum mechanicsintractablecomputational tasksCharacterizing quantum supremacy in near-term devicesherebutterfly effectqubitArguablyexponentialseparationcomputationalimprovementsclassicalalgorithmssimulate quantum circuitsincrease the sizesimulation costherecross-entropy benchmarkingfault-toleranthighly complex

Space-time volume of a quantum circuit computation. The computational cost for quantum simulation increases with the volume of the quantum circuit, and in general grows exponentially with the number of qubits and the circuit depth. For asymmetric grids of qubits, the computational space-time volume grows slower with depth than for symmetric grids, and can result in circuits exponentially easier to simulate.

A blueprint for demonstrating quantum supremacy with superconducting qubitsherecross-entropy benchmarkingdigital quantum processorsurface code error correction

Photograph of two gmon superconducting qubits and their tunable coupler developed by Charles Neill and Pedram Roushan.

quantum processorsquantum algorithmsapplications

Announcing Open Images V4 and the ECCV 2018 Open Images Challenge

Monday, April 30, 2018

Posted by Vittorio Ferrari, Research Scientist, Machine PerceptionOpen ImagesupdatingrefiningOpen Images V4largest existing datasetvisualizer

Annotated images from the Open Images dataset. Left: Mark Paul Gosselaar plays the guitar by Rhys A. Right: Civilization by Paul Downey. Both images used under CC BY 2.0 license.

Open Images Challenge2018 European Conference on Computer VisionPASCAL VOCImageNetCOCO

12.2M bounding-box annotations for 500 categories on 1.7M training images,

A broader range of categories than previous detection challenges, including new objects such as “fedora” and “snowman”.

In addition to the object detection main track, the challenge includes a Visual Relationship Detection track, on detecting pairs of objects in particular relations, e.g. “woman playing guitar”.

crowdsource.google.com

Google at ICLR 2018

Sunday, April 29, 2018

Posted by Jeff Dean, Google Senior Fellow, Head of Google Research and Machine Intelligence6th International Conference on Learning Representationsmachine learningneural networksdeep learningblueSenior Program Chair:Tara SainathSteering Committee includes:Hugo LarochelleOral ContributionsWasserstein Auto-EncodersIlya Tolstikhin, Olivier Bousquet, Sylvain Gelly, Bernhard ScholkopfOn the Convergence of Adam and Beyond(Best Paper Award)Sashank J. Reddi, Satyen Kale, Sanjiv KumarAsk the Right Questions: Active Question Reformulation with Reinforcement LearningChristian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, Wei WangBeyond Word Importance: Contextual Decompositions to Extract Interactions from LSTMsW. James Murdoch, Peter J. Liu, Bin YuConference PostersBoosting the Actor with Dual CriticBo Dai, Albert Shaw, Niao He, Lihong Li, Le SongMaskGAN: Better Text Generation via Filling in the _______William Fedus, Ian Goodfellow, Andrew M. DaiScalable Private Learning with PATENicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Ulfar ErlingssonDeep Gradient Compression: Reducing the Communication Bandwidth for Distributed TrainingYujun Lin, Song Han, Huizi Mao, Yu Wang, William J. DallyFlipout: Efficient Pseudo-Independent Weight Perturbations on Mini-BatchesYeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, Roger GrosseLatent Constraints: Learning to Generate Conditionally from Unconditional Generative ModelsAdam Roberts, Jesse Engel, Matt HoffmanMulti-Mention Learning for Reading Comprehension with Neural CascadesSwabha Swayamdipta, Ankur P. Parikh, Tom KwiatkowskiQANet: Combining Local Convolution with Global Self-Attention for Reading ComprehensionAdams Wei Yu, David Dohan, Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V. LeSensitivity and Generalization in Neural Networks: An Empirical StudyRoman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-DicksteinAction-dependent Control Variates for Policy Optimization via Stein IdentityHao Liu, Yihao Feng, Yi Mao, Dengyong Zhou, Jian Peng, Qiang LiuAn Efficient Framework for Learning Sentence RepresentationsLajanugen Logeswaran, Honglak Lee Fidelity-Weighted LearningMostafa Dehghani, Arash Mehrjou, Stephan Gouws, Jaap Kamps, Bernhard Schölkopf Generating Wikipedia by Summarizing Long SequencesPeter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, Noam ShazeerMatrix Capsules with EM RoutingGeoffrey Hinton, Sara Sabour, Nicholas Frosst Temporal Difference Models: Model-Free Deep RL for Model-Based ControlSergey Levine, Shixiang Gu, Murtaza Dalal, Vitchyr Pong Deep Neural Networks as Gaussian ProcessesJaehoon Lee, Yasaman Bahri, Roman Novak, Samuel L. Schoenholz, Jeffrey Pennington, Jascha Sohl-DicksteinMany Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence at Every StepWilliam Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M. Dai, Shakir Mohamed, Ian Goodfellow Initialization Matters: Orthogonal Predictive State Recurrent Neural NetworksKrzysztof Choromanski, Carlton Downey, Byron BootsLearning Differentially Private Recurrent Language ModelsH. Brendan McMahan, Daniel Ramage, Kunal Talwar, Li ZhangLearning Latent Permutations with Gumbel-Sinkhorn NetworksGonzalo Mena, David Belanger, Scott Linderman, Jasper Snoek Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement LearningBenjamin Eysenbach, Shixiang Gu, Julian Ibarz, Sergey LevineMeta-Learning for Semi-Supervised Few-Shot ClassificationMengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Josh Tenenbaum, Hugo Larochelle, Richard ZemelThermometer Encoding: One Hot Way to Resist Adversarial ExamplesJacob Buckman, Aurko Roy, Colin Raffel, Ian GoodfellowA Hierarchical Model for Device PlacementAzalia Mirhoseini, Anna Goldie, Hieu Pham, Benoit Steiner, Quoc V. Le, Jeff Dean Monotonic Chunkwise AttentionChung-Cheng Chiu, Colin RaffelTraining Confidence-calibrated Classifiers for Detecting Out-of-Distribution SamplesKimin Lee, Honglak Lee, Kibok Lee, Jinwoo ShinTrust-PCL: An Off-Policy Trust Region Method for Continuous ControlOfir Nachum, Mohammad Norouzi, Kelvin Xu, Dale SchuurmansEnsemble Adversarial Training: Attacks and DefensesFlorian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel Stochastic Variational Video PredictionMohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy Campbell, Sergey LevineDepthwise Separable Convolutions for Neural Machine TranslationLukasz Kaiser, Aidan N. Gomez, Francois CholletDon’t Decay the Learning Rate, Increase the Batch Size Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, Quoc V. LeGenerative Models of Visually Grounded ImaginationRamakrishna Vedantam, Ian Fischer, Jonathan Huang, Kevin MurphyLarge Scale Distributed Neural Network Training through Online DistillationRohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, Geoffrey E. HintonLearning a Neural Response Metric for Retinal ProsthesisNishal P. Shah, Sasidhar Madugula, Alan Litke, Alexander Sher, EJ Chichilnisky, Yoram Singer, Jonathon ShlensNeumann Optimizer: A Practical Optimization Algorithm for Deep Neural NetworksShankar Krishnan, Ying Xiao, Rif A. SaurousA Neural Representation of Sketch DrawingsDavid Ha, Douglas EckDeep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson SamplingCarlos Riquelme, George Tucker, Jasper SnoekGeneralizing Hamiltonian Monte Carlo with Neural NetworksDaniel Levy, Matthew D. Hoffman, Jascha Sohl-DicksteinLeveraging Grammar and Reinforcement Learning for Neural Program SynthesisRudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, Pushmeet Kohli On the Discrimination-Generalization Tradeoff in GANsPengchuan Zhang, Qiang Liu, Dengyong Zhou, Tao Xu, Xiaodong HeA Bayesian Perspective on Generalization and Stochastic Gradient DescentSamuel L. Smith, Quoc V. Le Learning how to Explain Neural Networks: PatternNet and PatternAttributionPieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, Sven DähneSkip RNN: Learning to Skip State Updates in Recurrent Neural NetworksVíctor Campos, Brendan Jou, Xavier Giró-i-Nieto, Jordi Torres, Shih-Fu ChangTowards Neural Phrase-based Machine TranslationPo-Sen Huang, Chong Wang, Sitao Huang, Dengyong Zhou, Li DengUnsupervised Cipher Cracking Using Discrete GANsAidan N. Gomez, Sicong Huang, Ivan Zhang, Bryan M. Li, Muhammad Osama, Lukasz Kaiser Variational Image Compression With A Scale HyperpriorJohannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, Nick JohnstonWorkshop PostersLocal Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter ValuesJulius Adebayo, Justin Gilmer, Ian Goodfellow, Been KimStoachastic Gradient Langevin Dynamics that Exploit Neural Network StructureZachary Nado, Jasper Snoek, Bowen Xu, Roger Grosse, David Duvenaud, James MartensTowards Mixed-initiative generation of multi-channel sequential structureAnna Huang, Sherol Chen, Mark J. Nelson, Douglas EckCan Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games?Maithra Raghu, Alex Irpan, Jacob Andreas, Robert Kleinberg, Quoc V. Le, Jon KleinbergGILBO: One Metric to Measure Them AllAlexander Alemi, Ian FischerHoME: a Household Multimodal EnvironmentSimon Brodeur, Ethan Perez, Ankesh Anand, Florian Golemo, Luca Celotti, Florian Strub, Jean Rouat, Hugo Larochelle, Aaron CourvilleLearning to Learn without LabelsLuke Metz, Niru Maheswaranathan, Brian Cheung, Jascha Sohl-DicksteinLearning via Social Awareness: Improving Sketch Representations with Facial FeedbackNatasha Jaques, Jesse Engel, David Ha, Fred Bertsch, Rosalind Picard, Douglas EckNegative Eigenvalues of the Hessian in Deep Neural Networks Guillaume Alain, Nicolas Le Roux, Pierre-Antoine ManzagolRealistic Evaluation of Semi-Supervised Learning AlgorithmsAvital Oliver, Augustus Odena, Colin Raffel, Ekin Cubuk, lan GoodfellowWinner's Curse? On Pace, Progress, and Empirical Rigor D. Sculley, Jasper Snoek, Alex Wiltschko, Ali RahimiMeta-Learning for Batch Mode Active LearningSachin Ravi, Hugo Larochelle To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model CompressionMichael Zhu, Suyog Gupta Adversarial SpheresJustin Gilmer, Luke Metz, Fartash Faghri, Sam Schoenholz, Maithra Raghu,,Martin Wattenberg, Ian GoodfellowClustering Meets Implicit Generative ModelsFrancesco Locatello, Damien Vincent, Ilya Tolstikhin, Gunnar Ratsch, Sylvain Gelly, Bernhard ScholkopfDecoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity TasksVitalii Zhelezniak, Dan Busbridge, April Shen, Samuel L. Smith, Nils Y. HammerlaLearning Longer-term Dependencies in RNNs with Auxiliary LossesTrieu Trinh, Quoc Le, Andrew Dai, Thang LuongGraph Partition Neural Networks for Semi-Supervised ClassificationAlexander Gaunt, Danny Tarlow, Marc Brockschmidt, Raquel Urtasun, Renjie Liao, Richard ZemelSearching for Activation FunctionsPrajit Ramachandran, Barret Zoph, Quoc LeTime-Dependent Representation for Neural Event Sequence PredictionYang Li, Nan Du, Samy BengioFaster Discovery of Neural Architectures by Searching for Paths in a Large ModelHieu Pham, Melody Guan, Barret Zoph, Quoc V. Le, Jeff DeanIntriguing Properties of Adversarial ExamplesEkin Dogus Cubuk, Barret Zoph, Sam Schoenholz, Quoc Le PPP-Net: Platform-aware Progressive Search for Pareto-optimal Neural ArchitecturesJin-Dong Dong, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, Min SunThe Mirage of Action-Dependent Baselines in Reinforcement LearningGeorge Tucker, Surya Bhupatiraju, Shixiang Gu, Richard E. Turner, Zoubin Ghahramani, Sergey LevineLearning to Organize Knowledge with N-Gram MachinesFan Yang, Jiazhong Nie, William W. Cohen, Ni LaoOnline variance-reducing optimizationNicolas Le Roux, Reza Babanezhad, Pierre-Antoine Manzagol

Blog

Labels

Archive

Feed