People on the autism spectrum often struggle to interpret facial cues, partially because they are unable to recognize how eye gaze direction is used to predict the actions and intentions of others. This can have huge implications for understanding behavior, language learning, and social interaction.

A Penn State study is developing a targeted intervention method using game technology designed to improve sensitivity to eye gaze cues, which could treat core symptoms of autism and help improve social skills.

The work appears in JCPP Advances, the Journal of Child Psychology and Psychiatry Advances, published under the Association for Child and Adolescent Mental Health.

The intervention, Social Games for Autistic Adolescents (SAGA), has been in development since 2013, when Suzy Scherf, associate professor of psychology and Social Science Research Institute co-funded faculty member, and her team were awarded a Collaborative Online International Learning (COIL) grant. “We wanted to develop a computer game intervention to improve sensitivity to eye gaze cues for individuals on the autism spectrum,” she said.

In 2015, the work was featured on the Big Ten Network’s LiveBig spotlight segment shown during football and basketball games. Jason Griffin, doctoral candidate in the department of psychology who works in the Laboratory of Developmental Neuroscience directed by Scherf, took feedback from participants who played SAGA during the filming of the segment to further refine the game.

During game play, participants progress through a narrative storyline and interact with animated characters using a range of nonverbal social cues, including eye gaze, to guide goal-directed behavior and solve problems in the game.

“We built a narrative about being a pet detective, and we told participants that they had to find the lost pet,” said Griffin. “While they are progressing through the game, they are learning how to respond to the nonverbal social cues from the humanoid figures that feature slightly enlarged heads and eyes.”

Scherf said the game strategy is a novel approach in this type of intervention, because skill learning is separate from the story line in the game. “Participants get to make their own choices in the game narrative, while the game technology automatically calibrates to their performance to keep them engaged and learning. This means that participants spend hours engaged in the eye gaze task, which would be very hard to get them to do otherwise.”

According to Scherf the game is a product of a Penn State cross-disciplinary collaboration, as a team of undergraduate and graduate visual arts students worked with the lab programmer to design game graphics.

“The eye gaze of each character had to be exactly right and mimic human motion, which was very tricky. Jason iteratively tested the game and gave feedback, so the team could understand what we wanted to do as scientists. It was a total collaborative effort that also fit into the resources and timeline constraints of the project.”

SAGA study results

During the study, 40 adolescents on the autism spectrum were randomized into either the treatment or standard care control group. Adolescents in the treatment group were asked to play SAGA for 30-minute sessions at home three times a week over 10 weeks.

According to Griffin, after the 10-week period adolescents in the treatment group developed increasing sensitivity to human eye gaze cues, whereas the standard care group did not. “Participants who experienced a sufficient dose of gameplay showed larger treatment-related improvements, such as the ability to perceive and interpret eye gaze cues from other people.”

The improvements to human eye gaze cues translated to outside of the game as well. “We showed participants pictures and movies of people and noted increases in eye gaze cues amongst our treatment participants,” said Scherf.

However, the study’s most relevant finding was that adolescents in the treatment group demonstrated improvements in social skills as documented by their parents. “We found that the participants who showed the largest improvement to understand human eye gaze cues in the game had corresponding improvements in social skills. Parents had no knowledge of their children’s performance in the game, so their observations were completely unbiased,” Scherf said.

Additionally, the researchers reported they were able to retain all participants throughout the study. “Often, interventions take place in a clinic, and it may be difficult to get kids to want to participate. The kids found the game very engaging and were excited to play it,” said Griffin.

Scherf is appreciative of Griffin’s commitment to the project, which spans over four years. “Jason was the only graduate student collecting data and he also coordinated a team of full-time staff and students. Jason also took the lead in the coordinating of testing, in which families came in every weekend for months.”

During the second phase of the project, researchers will develop improvements for the game to determine if skills learned translate into real-life conversations.

Other researchers on the project include SSRI co-funded faculty members Charles Greier, Dr. Frances Keesler Graham Early Career Professor and associate professor of human development and family studies; and Joshua Smyth, distinguished professor of biobehavioral health and SSRI associate director.

The work is being supported by a grant from the National Institute of Mental Health.