Glass Social Helper

A collaborated design project for Google Glass

This project focuses on adapting technology to a subset of the population with special needs—a subset of the population where technology has the most potential to impact positive change. Social Helper is an effort to transform the Google Glass platform into a dialogue-enhancing tool that enables people with Autism spectrum disorder (ASD) to lead more intuitive conversations for themselves.

Our product is a context-sensitive interface on Google Glass to provide the user with short descriptions of current conversational topics so that the user can communicate more effectively. Using the microphone on Google Glass, the application listens to the user’s conversations in order to determine the topic and the tone of conversation using machine learning algorithms. The application also uses the camera on Google Glass to capture the facial expression of the other speaker and analyze using computer vision algorithms to provide clues to the user.

Current autism social skills applications listed on the Autism Speaks website mainly work on mobile and tablet devices, which would not be as helpful when engaging in conversations with others.

There are currently very few social helper wearables for our user demographic. One product that has been developed is MIT’s artificial intelligence “social coach” that focuses on only tracking the conversational tone. While this is development is promising, we feel their product lacks the features such as facial expression recognition needed to enhance the quality of communication between people with ASD and others. In addition, with smartwatches, much attention is being diverted towards reading the results on the person’s wrist. Our product will be more direct since the visual cues will be displayed on the Google Glass, in the user’s line of vision.