Recently I read some papers about interaction design and technology for children with disabilities in physical, intellectual and mental aspects.

Assistive technology offers powerful possibilities to improve the learning efficiency of the disable. They include telecommunication devices, high-resolution monitors, speech digitizers and synthesizers, and electronic communication aids [5]. Requirements for diverse disabilities and special education are increasing, which means research and developments in this field have promising future.

This summary covers the interaction design in three aspects: physical, intellectual and mental disability, and compares related works.

1. Physical Disability

There are a lot of categories of physical disabilities, such as blindness, deaf which may be either congenital, or a result of injury, muscular dystrophy and so forth. Now a lot of technologies are in use for disabled children, for instance, touch screens, voice control, eye tracking etc, [5].

This summary focuses on the deaf or hard-hearing (d/hh) people. Some of them are born deaf, while others become deaf after language developed or because of the old age. For those are not completely deaf, devices such as sound amplifications help a lot. It is estimated that 95% of school-aged children born deaf have hearing parents, which means that they may not have access to sign language until they entering schools.

For the education of d/hh children who use sign language as their first language, researchers paid much attention on the speech-text system, which offers subtitles videos presentations, such as Rapidtext and C-Print [4].

C-Print is a software providing speech-to-text support services with embedded educational tools such as highlighting, note-taking and messaging in mainstream classes. The evaluation results show that students’ understanding of the content of classes via C-Print and interpreters seemed similar while the purpose and frequency of the two approaches had a significant difference. Generally, C-Print had positive effect on the students learning in classes.

However, the efficiency of these assistive software lies on the reading ability of children. But the average reading level of the deaf in the U.S is under fourth grade level, which is normalized against that of their hearing peers [9].

So perhaps a new method to facilitate education is needed for children who use sign language as the first language.

1.1 Head-Mounted Displays for deaf children

Head-Mounted displays (HMDs) were evaluated as a tool to facilitate student-

teacher interaction in sign language, which allows to keep signed narration in sight, not looking directly [6]. Because in normal deaf educational settings children must split attention between sign narration and the visual content.

The driving principle of this design is to deliver instruction in sign language rather than word captions for the young who are learning sign language as first language.

• Research Question

The purpose of the project is to explore the configuration of and potential benefits of head-mounted displays for education in American Sign Language (ASL).

• Method

In this project, 18 d/hh participants aging from 13 to 18 who are using ASL as native language completed two-phased evaluation. In the first phase, 8 participants were required to watch a video about astronomy through an HMD with ASL narration and give feedback. In the second phase, 10 participants watched a planetarium show, 5 of whom directly watched the dome with narration projected on it while the other 5 watched the narration through an HMD. Participants could adjust the size, position and brightness of narration with a video game controller.

• Result

The two phases of evaluation were recorded and coded into English transcript by a deaf interpreter. In the first phase of evaluation, 8 participants were divided into 2 focus groups and discussed the design, utility and comfort of the HMD. The same method was used to the second phase of evaluation.

According to the video record and transcript, primary themes included signer position, fit and split attention.

Most participants preferred right eye and positioned the signer on the top-right with one exception. During the process of watching the video, they tended to move the signer down to the center of the field of view.

Split attention most focused between the signer and the external video. More than a half participants gave negative comments and stated that they found it difficult to concentrate on two things at the same time.

The fit of the HMD was also negative. Most of the participants described the discomfort especially on weight and balance. The problems possibly lay on the fact that the hardware was for military and tactical use which is not suitable for teenagers.

• Conclusion

Two source of discomfort of HMD were identified in this project. The hardware was too large and bulky for children to use effectively, which indicated the future work on developing a smaller and lighter HMD. Besides, the reposition of signer to the center of the view field may minimize the visual attention subjects needed to shift between the narration and the external world.

2. Intellectual Disability

Intellectual Disability (ID) and Autism Disorder Spectrum (ASD) are inconsistently related so some research work is interlinked. Both ID and ASD can’t be cured and can last for years or lifelong.

2.1 KROG

KROG (Kinect-RObot for Gaming) was a project aimed to explore new spaces of interaction for children with Intellectual Development Disorder (IDD). The team adopted a game-based approach with the blending of two paradigms: full-body interaction and interaction with mobile robots [3].

In this paper, design challenges were outlined as well as requirements and guidelines on the field of both full-body interaction and mobile robot.

• Design requirements

To integrate the interaction among children, robot and the virtual environment, designers should master the complexity because both the robot and virtual world would give feedback based on children’s actions. The team gave 12 design requirements for the affordances of the blending technology.

Generally, these requirements can be divided based on the purpose, methods and general guideline. For the purpose of this project, the system should get familiarity and trust of the children and focus on virtual–physical, real–imaginative world and imaginative–real world transition. In the process of the games, the system should give children feedback, prompt, instruction, reward and restriction. The system should serve for the social mediation of the children, which ought to encourage them to interact with others rather than only with the robot.

• Prototype

The virtual environment consisted of simple colored shapes, which are the mirror images of children and robot’s silhouettes.

The robot is called Teo, which had an egg-line shape and made of fabric. Children can personalize its expression by sticking face components. It had several buttons on its head for children to press when manipulating games.

Teo offered a variety of visual, sensory and spatial stimuli. It can give out colored LED light; it can vibrate, rotate and move around with omni-wheels,. With sensors and actuators, Teo had different statement such as waiting, invitation to interact, angry and scared.

With the assistance of virtual world and the robot, children would be required to complete a set of structured learning tasks inspired by therapeutic approaches, only when the children were willing to do so. Before the tasks, the child would generally get familiar with Teo and the virtual environment under the instruction of caregivers. Teo would move following the child, speak to him/ her, controlled by an operator with a remote controller.

Then four games were introduced to the child. These structured learning activities included simple choice making and recognition tasks, physical games, storytelling and the farewell greeting. For example, “Witch says colors” was a game requiring child and Teo move to correct space according to the image on the screen. The child might reach the position but Teo not, so that the child would be asked for help.

• Result

The result was quite positive that the project elicited operational behaviors, social interaction and emotional responses the normally do not occur by traditional methods such as the willingness to do the tasks and emotional calmness during the interaction with robots.

2.2 Summary

Children with ID and ASD have similar spectrums to some extent, so some therapeutic technologies interlink. KROG project developed quite maturely, blending motion-based touchless technology via Kinect and the robot.

In the last summary, robot technologies were introduced such as Auti [6]. Compared with Auti, Teo is more functional due to the personalized expression, movement and voice feedback. However, strictly speaking Teo is not an independent robot because most of its behavior is controlled by an operator.

Also in the last summary, ECHO [2] for children with ASC is also a maturely developed system. The paper offered some initial recommendations for the design of virtual environment, part of which were the same with those in KROG, such as the familiarity and resolvability.

3. Mental Disability

Mind Full is a neuro-feedback (also called EEG biofeedback) system and application developed by School of Interactive Arts & Technology, Simon Fraser University in Canada, to help vulnerable children in Pokahra, Nepal [1]. Most of these children have trauma caused by poverty, parental mental illness, homelessness, domestic violence, etc. Mind Full was designed to help them learn and practice self-regulation through simple games on a tablet connected to a EEG headset. In this project, one of the biggest challenge is that the children have no computer knowledge and can’t speak English, most of them can’t read or write, which means the games should be easy to understand graphically.

3.1 Research question

Before the experiment, the researchers put forward an overarching research question, “What is needed when helping children in poverty learn self-regulation?”, or “How to help children in poverty success?”.

The result of current pediatric psychotherapy shows that mindfulness practices is an effective way to improve executive functioning and limbic system after trauma. [8] Mindfulness training includes breathing, relaxation and attention exercise, which is a kind of therapeutic intervention working on both children and adults. Then the researchers listed the following main research questions:

1. Can children in poverty learn neuro-feedback based tablet games quickly? ​
2. If so, can they use it to learn and practice self-regulation?
3. Can children transfer this control to other context (real life)?
4. Does EEG(electroencephalography) help?

Among these, the third research question was the most important one. If the children can’t transfer their learning into daily life, the project would not achieve its main objective.

3.2 Technology Development

• Hardware

In recent years, the decrease of the price of consumer-grade EEG systems resulted in the popularity of brain computer interface (BCI). This project utilized one of the most commonly used EEG headset NeuroSky and selected Mindwave Mobile, a wireless version, as the hardware for experiment.

The research team developed a secondary application that ran on the second tablet which could display the brainwave and enable therapist to control the game difficulty on-the-fly.

• The system

The main principle of the system was to encourage children to perform physical actions that would shift the child’s physiology and corresponding brainwave state to teach them how to self-regulate around relaxation or anxiety and focus or attention.

• Games

Three simple games were designed for these children: Pinwheel, Paraglider and Moving stones. The main principle of the game design was based on familiar activities from children’s everyday lives to encourage children to perform physical actions that shifted their physiology and corresponding brainwave. In this way, children could learn how to self-regulate around relaxation/ anxiety and focus/ attention. The second principle was that the UI design should be easy to understand and manipulate and the third one was that an on-the-fly calibration system was necessary so that consolers could calibrate while the children were playing the games.

 

 

3.3 Method

In the field study, twenty-two girls with trauma were divided into two groups. The first group used Mind Full about 10 minutes per day, under the instruction of a counselors for three to five times a week in six-week periods. All the participants have been assessed before the study by the Western therapist. Four counselors working in the school were trained and supervised by Western therapists. All the girls were re-assessed at the end of six weeks.

The assessments were completed in the form of open survey questions, statements rated using an interval five point Likert scale and space for comments that explained ratings.

3.4 Result and Discussion

The research team compared the two assessments of treatment and control group to get preliminary results. The results were as follows:

• Can children in poverty learn neuro-feedback based tablet games quickly?

All the girls quickly understood the games.

• If so, can they use it to learn and practice self-regulation?

For easy game like Pinwheel, children can complete without support. But for harder games like Stone, some children need minor re-calibration can encouragement.

• Can children transfer this control to other context (real life)?

Calmness and attention increased in classes.

Negative behaviors decrease in second assessment for treatment group.

• Does EEG(electroencephalography) help?

Most counselors were positive about Mind-Full. Children felt like it and it was easier for them to focus on classes.

Apart from the results above, there are some unexpected benefit​s. For example, one girl who had serve difficulties in concentrating performed well in games. So that counselors determined to change her counselling sessions. Another girl who appeared outwardly calm but had trouble with learning was diagnosed with learning disability. In the experiment, consolers found that she had a great difficulty with the Paraglider game and the brainwave showed that she was in great stress. After investigating her family condition, the consolers determined to change their assessment of her as learning disabled.

 

Mind-Full received overwhelmingly positive result. But this was only the short-time result and it remains to see whether the improvement will be hold. For further work, they would focus on helping children with different levels of trauma. And the long term goal is to develop a training program that will involve the dissemination of additional system to work with more orphanages and schools throughout Nepal and eventually the world.

3.5 Summary

There are some similarities in this project and Lilypad. Mind-Full and the student version of Lilypad, Tadpole, both designed an application​ for regulation​. But their methods for self-regulation are different. Tadpole converts the assessment system into a self-regulated one. Mind-Full uses the western and traditional Buddhist therapeutic methods. Tadpole is more functional while Mind-Full focuses more on games.

Reference:

1. Alissa N. Antle, Leslie Chesick, Aaron Levisohn, Srilekha Kirshnamachari Sridharan, and Perry Tan. 2015. Using neurofeedback to teach self-regulation to children living in poverty. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC ’15). ACM, New York, NY, USA, 119-128. DOI=http://dx.doi.org/10.1145/2771839.2771852
2. Alyssa M. Alcorn, Helen Pain, and Judith Good. 2014. Motivating children’s initiations with novelty and surprise: initial design recommendations for autism. In Proceedings of the 2014 conference on Interaction design and children (IDC ’14). ACM, New York, NY, USA, 225-228. DOI=http://dx.doi.org/10.1145/2593968.2610458
3. Andrea Bonarini, Francesco Clasadonte, Franca Garzotto, and Mirko Gelsomini. 2015. Blending robots and full-body interaction with large screens for children with intellectual disability. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC ’15). ACM, New York, NY, USA, 351-354. DOI=http://dx.doi.org/10.1145/2771839.2771914
4. Elliot, L., Stinson, M., Easton, D., & Bourgeois, J. (2008, March 25). College students’ learning with C-Print’s educational software and automatic speech recognition. Paper presented at the American Educational Research Association Annual Meeting, New York City, NY.
5. Garrick Duhaney, L.,M., & Duhaney, D. C. (2000). Assistive technology: Meeting the needs of learners with disabilities. International Journal of Instructional Media, 27(4), 393.
6. Helen E. Andreae, Peter M. Andreae, Jason Low, and Deidre Brown. 2014. A study of auti: a socially assistive robotic toy. In Proceedings of the 2014 conference on Interaction design and children(IDC ’14). ACM, New York, NY, USA, 245-248.
7. Michael Jones, M. Jeannette Lawler, Eric Hintz, Nathan Bench, Fred Mangrubang, and Mallory Trullender. 2014. Head mounted displays and deaf children: Facilitating Sign Language in Challenging Learning Environments. In Proceedings of the 2014 conference on Interaction design and children (IDC ’14). ACM, New York, NY, USA, 317-320.
8. Lee, J., Semple, R.J., Rose, D. Miller, L. Mind-Fullness- based cognitive therapy for children: Results of a pilot study, Journal of Cognitive Psychotherapy, 22, 1 (2008), 15-28.
9. Qi and R. E. Mitchell. Large-scale academic achievement testing of deaf and hard-of-hearing students: Past, present, and future. Journal of Deaf Studies and Deaf Education, 2011.