Google AI Blog: September 2016

Introducing the Open Images Dataset

Friday, September 30, 2016

Posted by Ivan Krasin and Tom Duerig, Software EngineersComputer Visionautomatically caption imagesnatural language replies in response to shared photosImageNetCOCOYFCC100MOpen ImagesCreative Commons Attribution^*Google Cloud Vision API

Annotated images form the Open Images dataset. Left: Ghost Arches by Kevin Krejci. Right: Some Silverware by J B. Both images used under CC BY 2.0 license

DeepDreamartistic style transferOpen Imagesrecently released YouTube-8M
* While we tried to identify images that are licensed under a Creative Commons Attribution license, we make no representations or warranties regarding the license status of each image and you should verify the license for each image yourself.^↩

Image Compression with Neural Networks

Thursday, September 29, 2016

Posted by Nick Johnston and David Minnen, Software EngineersFull Resolution Image Compression with Recurrent Neural Networksprevious researchimage recognitiontext summarizationreleasing our compression modelTensorFlowGated Recurrent UnitRNNDeep Residual Learning for Image Recognition

The initial residual, R[0], corresponds to the original image I: R[0] = I.

Set i=1 for to the first iteration.

Iteration[i] takes R[i-1] as input and runs the encoder and binarizer to compress the image into B[i].

Iteration[i] runs the decoder on B[i] to generate a reconstructed image P[i].

The residual for Iteration[i] is calculated: R[i] = I - P[i].

Set i=i+1 and go to Step 3 (up to the desired number of iterations).

Left: Original image, I = R[0]. Center: Reconstructed image, P[1]. Right: the residual, R[1], which represents the error introduced by compression.

The second pass through the network. Left: R[1] is given as input. Center: A higher quality reconstruction, P[2]. Right: A smaller residual R[2] is generated by subtracting P[2] from the original image.

Japanese ChinMS-SSIM

Left: Original image (1419 KB PNG) at ~1.0 MS-SSIM. Center: JPEG (33 KB) at ~0.9 MS-SSIM. Right: Residual GRU (24 KB) at ~0.9 MS-SSIM. This is 25% smaller for a comparable image quality

blocking artifacts

Left: Original. Center: JPEG. Right: Residual GRU.

our paper

Announcing YouTube-8M: A Large and Diverse Labeled Video Dataset for Video Understanding Research

Wednesday, September 28, 2016

Posted by Sudheendra Vijayanarasimhan and Paul Natsev, Software EngineersImageNetdetecting and classifying objects in static imagesVideo analysishelped re-imagine the photos experienceYouTube-8MKnowledge GraphSports-1Morder of magnitude increaseand

technical reportdataset browser

A dataset explorer allows browsing and searching the full vocabulary of Knowledge Graph entities, grouped in 24 top-level verticals, along with corresponding videos. This screenshot depicts a subset of dataset videos annotated with the entity “Guitar”.

The distribution of videos in the top-level verticals illustrates the scope and diversity of the dataset and reflects the natural distribution of popular YouTube videos.

featuresInception-V3 image annotation modelTensorFlowActivityNet,Sports-1Mtechnical report

A Neural Network for Machine Translation, at Production Scale

Tuesday, September 27, 2016

Posted by Quoc V. Le & Mike Schuster, Research Scientists, Google Brain Teamlaunch of Google TranslatePhrase-Based Machine Translationspeech recognitionimage recognitionGoogle’s Neural Machine Translation System: Bridging the Gap between Human and Machine TranslationRecurrent Neural Networks

Data from side-by-side evaluations, where human raters compare the quality of translations for a given source sentence. Scores range from 0 to 6, with 0 meaning “completely nonsense translation”, and 6 meaning “perfect translation."

An example of a translation produced by our system for an input sentence sampled from a news site. Go here for more examples of translations for input sentences sampled randomly from news sites and books.

TensorFlowTensor Processing UnitsAcknowledgements:Google Brain teamGoogle Translate teamBig Picture teamReferences:Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine TranslationYonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean. Technical Report, 2016.Sequence to Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le. Advances in Neural Information Processing Systems, 2014.Addressing the rare word problem in neural machine translationMinh-Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Wojciech Zaremba. Proceedings of the 53th Annual Meeting of the Association for Computational Linguistics, 2015.Neural Machine Translation by Jointly Learning to Align and TranslateDzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. International Conference on Learning Representations, 2015.Japanese and Korean voice searchMike Schuster, and Kaisuke Nakajima. IEEE International Conference on Acoustics, Speech and Signal Processing, 2012.Neural Machine Translation of Rare Words with Subword UnitsRico Sennrich, Barry Haddow, Alexandra Birch. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.

Show and Tell: image captioning open sourced in TensorFlow

Thursday, September 22, 2016

Posted by Chris Shallue, Software Engineer, Google Brain TeamGoogle Brain teammachine learning system to automatically produce captions that accurately describe imagesMicrosoft COCO 2015 image captioning challengeavailable as an open source modelTensorFlowShow and Tell: Lessons learned from the 2015 MSCOCO Image Captioning ChallengeIEEE Transactions on Pattern Analysis and Machine Intelligence

Automatically captioned by our system.

So what’s new?Inception V1Inception V2Inception V3fine-tuningclassifydescribehere

Left: the better image model allows the captioning model to generate more detailed and accurate descriptions. Right: after fine-tuning the image model, the image captioning system is more likely to describe the colors of objects correctly.

DistBelief software framework

When the model is presented with scenes similar to what it’s seen before, it will often re-use human generated captions.

does indeed

Our model generates a completely new caption using concepts learned from similar scenes in the training set.

hererecently released Inception-ResNet-v2 model

The 280-Year-Old Algorithm Inside Google Trips

Tuesday, September 20, 2016

Posted by Bogdan Arsintescu, Software Engineer & Sreenivas Gollapudi, Kostas Kollias, Tamas Sarlos and Andrew Tomkins, Research Scientists
Algorithms Engineeringannounced Google TripsLeonhard Eulerbeautiful mathematical paper

Image from Wikipedia

Geometriam SitusGraph Theory

Image from Wikipedia

OrienteeringapproximatelyTravelling Salesman ProblemAlgorithms for Travel Itinerariestriangle inequalityalgorithm discovered by Christofides

We start with all our destinations separate, and repeatedly connect together the closest two that aren’t yet connected. This doesn’t yet give us an itinerary, but it does connect all the destinations via a minimum spanning tree of the graph.

We take all the destinations that have an odd number of connections in this tree (Euler proved there must be an even number of these), and carefully pair them up.

Because all the destinations now have an even number of edges, we’ve created an Eulerian graph, so we create a route that crosses each edge exactly once.

We now have a great route, but it might visit some places more than once. No problem, we find any double visits and simply bypass them, going directly from the predecessor to the successor.

Construction of an Eulerian Tour in a location graph

Itineraries in Google TripsBetter Itineraries Through the Wisdom of Crowdsestimate delays on highwayswhen restaurants are most busywhen places are popularChanging of the GuardAndroidiOS

The 2016 Google Earth Engine User Summit: Turning pixels into insights

Monday, September 19, 2016

Posted by Chris Herwig, Program Manager, Google Earth EngineWe are trying new methods [of flood modeling] in Earth Engine based on machine learning techniques which we think are cheaper, more scalable, and could exponentially drive down the cost of flood mapping and make it accessible to everyone."Cloud to Street
Google Earth Engine User Summit 2016

Earth Engine founder Rebecca Moore kicking off the first day of the summit

twenty-five hands-on workshops

Terra Bellasession on using SkySat imagery

Cross-correlation between Landsat 8 NDVI and the sum of CHIRPS precipitation. Red is high cross-correlation and blue is low. The gap in data is because CHIRPS is masked over water.

a time series sessionLandsat 8 NDVICHIRPS precipitationr-selectedworkshop sessionSentinel 2A

Plenary Speakers

Agriculture in the Sentinel era: scaling up with Earth Engine, Guido Lemoine, European Commission's Joint Research Centre

Flood Vulnerability from the Cloud to the Street (and back!) powered by Google Earth Engine, Beth Tellman, Arizona State University and Cloud to Street

Accelerating Rangeland Conservation, Brady Allred, University of Montana

Monitoring Drought with Google Earth Engine: From Archives to Answers, Justin Huntington, Desert Research Institute / Western Regional Climate Center

Automated methods for surface water detection, Gennadii Donchytes, Deltares

Lightning Presentations

Mapping the Behavior of Rivers, Alex Bryk, University of California, Berkeley

Climate Data for Crisis and Health Applications, Pietro Ceccato, Columbia University

Appalachian Communities at Risk, Matt Wasson, Jeff Deal, Appalachian Voices

Water, Wildlife and Working Lands, Patrick Donnelly, U.S. Fish and Wildlife Service

Stream-side NDVI and The Salmonid Population Viability Project, Kurt Fesenmyer, Trout Unlimited

Mapping Evapotranspiration for Water Use and Availability, Mac Friedrichs, USGS

Dynamic Wildfire Modeling in Earth Engine, Miranda Gray, Conservation Science Partners

Fishing at Scale, now also in Earth Engine, David Kroodsma, Skytruth

Mapping crop yields from field to national scales in Earth Engine, David Lobell, Stanford University

Mapping Pacific Wildfires Impacts with Earth Engine, Matthew Lucas, University of Hawaii

EarthEnv.org - Environmental layers for accessing status and trends in biodiversity, ecosystems and climate, Jeremy Malczyk, Map of Life

Building a Landsat 8 Mosaic of Antarctica, Allen Pope, University of Colorado Boulder

Monitoring Primary Production at Broad Spatial and Temporal Scales, Nathaniel Robinson, University of Montana

Assessing Urbanization Trends for Public Health: Modelling Nighttime Lights Imagery in Africa with Earth Engine, David Savory, University of California, San Francisco

National-scale mapping of forest carbon, Ty Wilson, US Forest Service

Utilizing Google Earth Engine to Enhance Decision-Making Capabilities, Brittany Zajic, NASA DEVELOP National Program

Keeping our users first

Research from VLDB 2016: Improved Friend Suggestion using Ego-Net Analysis

Thursday, September 15, 2016

Posted by Alessandro Epasto, Research Scientist, Google Research NY42nd International Conference on Very Large Data BasesEgo-net Community Mining Applied to Friend SuggestionSilvio LattanziVahab MirrokniAhmed Taei,myself,graphnumber of common friends

Figure 1: Ego-net of Sally.

ego-netthoseABCUVWABCUVWU,VWUWABCUVWdistributed systems

Figure 2: Clustering of the ego-net of Sally.

ABCUVWonlyUWBUnot

Computational Thinking from a Dispositions Perspective

Tuesday, September 13, 2016

Posted by Chris Stephenson, Head of Computer Science Education Programs at Google, and Joyce Malyn-Smith, Managing Project Director at Education Development Center (EDC)
(Cross-posted on the Google for Education Blog)Valerie Barr and ICT is an approach to solving problems in a way that can be implemented with a computer. Students become not merely tool users but tool builders. They use a set of concepts, such as abstraction, recursion, and iteration, to process and analyze data, and to create real and virtual artifacts. CT is a problem solving methodology that can be automated and transferred and applied across subjects.

personal qualities or soft skills needed for employmentsee full report from Economist Intelligence Unit here
Cuoco, Goldenberg, and Mark’sPotter and Vickers

International Society for Technology in EducationComputer Science Teachers Association

confidence in dealing with complexity,

persistence in working with difficult problems,

the ability to handle ambiguity,

the ability to deal with open-ended problems,

setting aside differences to work with others to achieve a common goal or solution, and

knowing one's strengths and weaknesses when working with others.

Post in the classroom a list of the Dispositions Leading to Success,

Help familiarize students with these dispositions by using the terms when talking with students and referring to the work they are doing. “Today we are going to be solving an open-ended problem. What do you think that means?”

Help students understand that they are developing these dispositions by congratulating them when these dispositions lead to success: “Great problem-solving skills!”; “Great job! Your persistence helped solve the problem”; “You dealt with ambiguity really well!”.

Engage students in discussions about the dispositions: “Today we are going to work in teams. What does it mean to be on a team? What types of people would you want on your team and why?”

Help students articulate their dispositions when developing their resumes or preparing for job interviews.

Integrate the importance of dispositions into their talks with students: examples of the problems they have solved, how the different skills of team members led to different solutions, the role persistence played in solving a problem/developing a product or service…

Talk about the importance of dispositions to employers and how they contribute to their own organizational culture, the ways employers ask interviewees about their dispositions or how interviewees might respond (e.g. use the terms and give examples).

recent blog postIf we can make these explicit connections for students, they will see how the devices and apps that they use everyday are powered by algorithms and programs. They will learn the importance of data in making decisions. They will learn skills that will prepare them for a workforce that will be doing vastly different tasks than the workforce of today.

Announcing the First Annual Global PhD Fellowship Summit and the 2016 Google PhD Fellows

Wednesday, September 7, 2016

Posted by Michael Rennaker, Program Manager, University RelationsPhD Fellowship ProgramAustraliaChina and East AsiaIndiaNorth AmericaEurope and the Middle EastJeff DeanFrançoise BeaufaysPeter NorvigMaya GuptaAmin Vahdat

Senior Google Fellow Jeff Dean shares how TensorFlow is used at Google

Dan RussellKristen LeFevreDouglas EckFrançoise BeaufaysMaggie Johnson

(Left) PhD Fellows share their work with Google researchers during the poster session
(Right) Research panelists share their journeys through academia and industry

2016 Global PhD FellowsComputational NeurosciencePrinceton UniversityColumbia UniversityThe University of EdinburghHuman-Computer InteractionUniversity of MelbourneUniversity of Southern CaliforniaStanford UniversityMassachusetts Institute of TechnologyMachine LearningSwiss Federal Institute of Technology ZurichUniversity of WashingtonNew York UniversityUniversity of MelbourneUniversity of OxfordKorea Advanced Institute of Science and TechnologyIndian Institute of Technology KharagpurTechnion - Israel Institute of TechnologyÉcole Polytechnique Fédérale de LausanneUniversity of Massachusetts, AmherstMassachusetts Institute of TechnologyThe Hong Kong University of Science and TechnologyPurdue University, West LafayetteMachine Perception, Speech Technology and Computer VisionRWTH Aachen UniversityUniversity of Michigan, Ann ArborUniversity of North Carolina at Chapel HillUniversity College LondonSwiss Federal Institute of Technology ZurichShanghai Jiao Tong UniversityCarnegie Mellon UniversityUniversity of QueenslandMarket AlgorithmsUniversity of Maryland, College ParkSaarland University - Saarbrucken GSCSCornell UniversityGeorgia Institute of TechnologyMobile ComputingUniversity of WisconsinCornell UniversityKorea Advanced Institute of Science and TechnologyUniversity of Texas, AustinNatural Language ProcessingRWTH Aachen UniversitySapienza - Università di RomaPrivacy and SecurityUniversity of New South WalesUniversity of Maryland, College ParkPennsylvania State UniversityÉcole Polytechnique Fédérale de LausanneUniversity of WisconsinProgramming Languages, Algorithms and Software EngineeringUniversity of OxfordIndian Institute of ScienceStructured Data and Database ManagementUniversity of Illinois, Urbana-ChampaignSystems and NetworkingTsinghua UniversityBrown UniversityIndian Institute of Technology BombayUniversity of CambridgeUniversity of California, BerkeleyThe Hong Kong University of Science and Technology

Reproducible Science: Cancer Researchers Embrace Containers in the Cloud

Tuesday, September 6, 2016

Posted by Dr. Kyle Ellrott, Oregon Health and Sciences University, Dr. Josh Stuart, University of California Santa Cruz, and Dr. Paul Boutros, Ontario Institute for Cancer ResearchToday we hear from the principal investigators of the ICGC-TCGA DREAM Somatic Mutation Calling Challenges about how they are encouraging cancer researchers to make use of Docker and Google Cloud Platform to gain a deeper understanding of the complex genetic mutations that occur in cancer, while doing so in a reproducible way.– Nicole Deflaux and Jonathan Bingham, Google GenomicsReproducibleCancer MoonshotBreak down silos and bring all the cancer fighters togethercontainersOntario Institute for Cancer ResearchUniversity of California Santa CruzSage BionetworksOregon Health and Sciences UniversityDockerCancer Cloud Pilots funded by the National Cancer InstituteICGC-TCGA DREAM Somatic Mutation Calling (SMC) ChallengesGoogle Cloud PlatformDNA somatic mutationsproviding key insightscontainerizationSMC-Het ChallengeSMC-RNA Challenge

SMC-Het Challenge: Tumour biopsies are composed of many different cell types in addition to tumour cells, including normal tissue and infiltrating immune cells. Furthermore, the tumours themselves are made of a mixture of different subpopulations, all related to one another through cell division and mutation. Critically, each sub-population can have distinct clinical outcomes, with some more resistant to treatment or more likely to metastasize than others. The goal of the SMC-Het Challenge is to identify the best methods for predicting tumor subpopulations and their “family tree” of relatedness from genome sequencing data.

SMC-RNA Challenge: The alteration of RNA production is a fundamental mechanism by which cancer cells rewire cellular circuitry. Genomic rearrangements in cancer cells can produce fused protein products that can bestow Frankenstein-like properties. Both RNA abundances and novel fusions can serve as the basis for clinically-important prognostic biomarkers. The SMC-RNA Challenge will identify the best methods to detect such rogue expressed RNAs in cancer cells.

SMC-HetSMC-RNA

Blog