Blog

The latest news from Google AI

The Machine Learning Behind Android Smart Linkify

Thursday, August 9, 2018

Posted by Lukas Zilka, Software Engineer, Google AI, Zürichwe launched Android 9 PieSmart Linkify

Smart Text SelectionTextClassifier APITensorFlowTensorFlow LiteFlatBuffershereFinding EntitiesConfirmation number: 857-555-3556feedforward neural networks

For the given text string, the first network assigns low scores to non-entities and a high score for the candidate that correctly selects the whole phone number.

And call 857 555 3556tomorrow.857 555 3556And call 857 555 3556 tomorrowAnd

And call 857 555 3556 tomorrow.857 555 3556And call 857 555 3556 tomorrowAndTextual Features

Given a candidate entity span, we extract: Left context: five words before the entity, Entity start: first three words of the entity, Entity end: last three words of the entity (they can be duplicated with the previous feature if they overlap, or padded if there are not that many), Right context: five words after the entity, Entity content: bag of words inside the entity and Entity length: size of the entity in number of words. They are then concatenated together and fed as an input to the neural network.

Character N-grams. Instead of using the standard word embedding technique for representing words, which keeps a separate vector for each word in the model and thus would be infeasible for mobile devices because of their large storage size, we use the hashed charactergram embedding. This technique represents the word as a set of all character subsequences of certain length. We use lengths 1 to 5. These strings are additionally hashed and mapped to a fixed number of buckets (see here for more details on the technique). As a result, the final model only stores vectors for each of the hash buckets, not each word/character subsequence, and can be kept small. The embedding matrix for the hashed charactergrams that we use has 20,000 buckets and 12 dimensions.

A binary feature that indicates whether the word starts with a capital letter. This is important for the network to know because the capitalization in postal addresses is quite distinct, and helps the networks to discriminate.

A Training DatasetSchema.orgConfirmation number:ID:Making it Work

Quantizing the embedding matrix to 8 bits. We found that we could reduce the size of the model almost 4x without compromising the performance, by quantizing the embedding matrix values to 8-bit integers.

Sharing embedding matrices between the selection and classification networks. This brings almost no loss and makes the model 2x smaller.

Varying the size of the context before/after the entities. On mobile screens text is often short, with not enough context, so the network needs to be exposed to this during training as well.

Creating artificial negative examples out of the positive ones for the classification network. For example for the positive example: “call me 857 555-3556 today” with a label “phone” we generate “call me 857 555-3556 today” as a negative example with a label “other”. This teaches the classification network to be more precise about the entity span. Without doing this, the network would be merely a detector whether there is a phone number somewhere in the input, regardless of the span.