Blog

The latest news from Google AI

Enabling E-Textile Microinteractions: Gestures and Light through Helical Structures

Friday, May 15, 2020

Posted by Alex Olwal, Research Scientist, Google Research

A scalable interactive E-textile architecture with embedded touch sensing, gesture recognition and visual feedback.

E-textile MicrointeractionsACM CHI 2020previously introduced E-textile architectureACM UIST 2018video about E-textile microinteractionsvideo about the E-textile architecture

E-textile microinteractions combining continuous sensing with discrete motion and grasps.

The Helical Sensing Matrix (HSM)capacitive sensinghelical sensing matrixtransmitreceive

Left: A Helical Sensing Matrix based on a 4×4 braid (8 conductive threads spiraled around the core). Magenta/cyan are conductive yarns, used as receive/transmit lines. Grey are passive yarns (cotton). Center: Flattened matrix, that illustrates the infinite number of 4×4 matrices (colored circles 0-F), which repeat along the length of the cord. Right: Yellow are fiber optic lines, which provide visual feedback.

Rotation Detection

Rotation is deduced from horizontal finger motion across the columns. The plots below show the relative capacitive signal strengths, which change with finger proximity.

Interaction Techniques and Design Guidelines

Simple gestures. We design for short interactions where the user either makes a single discrete gesture or performs a continuous manipulation.

Closed-loop feedback. We want to help the user discover functionality and get continuous feedback on their actions. Where possible, we provide visual, tactile, and audio feedback integrated in the device.

Our e-textile enables interaction based on capacitive sensing of proximity, contact area, contact time, roll, and pressure.

Braided fiber optics strands create the illusion of directional motion.

Motion Gestures (Flicks and Slides) and Grasping Styles (Pinch, Grab, Pinch)

Gesture elicitation study with imagined touch sensing.

Twelve participants (horizontal axis) performed 9 repetitions (animation) for the eight gestures (vertical axis). Each sub-image shows 16 overlaid feature vectors, interpolated to 80 observations over time.

User-Independent, Continuous Twist: Quantifying Precision and Speeduser-independent

Left: Weighted average subjective feedback. We mapped the 7-point Likert scale to a score in the range [-3, 3] and multiplied by the number of times the technique received that rating, and computed an average for all the scores. Right: Mean completion times for target distances show that Buttons were consistently slower.

Gesture Prototypes: Headphones, Hoodie Drawstrings, and Speaker Cord

Left: Tap = Play/Pause; Center: Double-tap = Next track; Right: Roll = Volume +/-

Interactive speaker cord for simultaneous use of continuous (twisting/rolling) and discrete gestures (pinch/pat) to control music playback.

Conclusions and Future DirectionsAcknowledgementsThis work is a collaboration across multiple teams at Google. Key contributors to the project include Alex Olwal, Thad Starner, Jon Moeller, Greg Priest-Dorman, Ben Carroll, and Gowa Mainini. We thank the Google ATAP Jacquard team for our collaboration, especially Shiho Fukuhara, Munehiko Sato, and Ivan Poupyrev. We thank Google Wearables, and Kenneth Albanowski and Karissa Sawyer, in particular. Finally, we would like to thank Mark Zarich for illustrations, Bryan Allen for videography, Frank Li for data processing, Mathieu Le Goc for valuable discussions, and Carolyn Priest-Dorman for textile advice.