Accessibility
Ethnographic research
Android app development
Enhancing messaging for the visually impaired
Timeline
14 weeks (Oct 2022 - Jan 2023)
Team
2 UX researchers
My role
UX researcher and app developer
Tools
Figma, Google Forms and Flutter
In a nutshell
Smooth screens pose a challenge for the visually impaired
“Messaging becomes really difficult sometimes”
17
research articles
18
user interviews
Development and feedback
Adding phonics to guide the users
“Wow, this made my life easier!”
Impact
India HCI 2023
Publication
72.2%
Adoption rate
Background
There is limited research in the domain of accessibility for the visually impaired.
With an inspiration to work in the domain for accessibility, we began identifying the various problems faced by the disabled, narrowing down the focus on visually impaired. This area of research has a lot of potential to be explored and the ever-evolving technology keeps increasing the demand for research in this area.
Objective
Despite many accessibility features, visually impaired (VI) users find it difficult to navigate smooth screens.
Although accessibility features like voice assistance, magnification, color contrast, etc. exist, visually impaired users still find it difficult to navigate through smooth smartphone screens. An optimal keyboard layout needs to be designed to help the target group use smartphones effectively.
Literature review
Reviewed 17 research papers on keyboard layouts and techniques used by VI users.
A total of seventeen papers were reviewed exploring various keyboard layouts and different techniques in which VI users communicate via messaging.
The major issue with keyboard layouts like QWERTY was the lack of efficiency due to proximity of keys.
Users could correlate the 9 and 12-key layouts with feature phones and found it easier to navigate through those keys.
Braille keyboards posed a limitation of knowledge of Braille and only the users who were well versed could use Braille to type in messages. Additionally, the received reply could not be translated to Braille, making the process complex.
Voice assistants and screen readers were also studied, showing a high dependency on these systems for the VI users.
User research
Conducted user interviews with 18 visually impaired participants with varying levels of visual difficulty.
After reviewing the existing literature, we decided to interact with the target group and understand their difficulties better. Interviews were scheduled both in-person at blind schools in Pune, India as well as via online calls.
A total of 18 participants were interviewed. Participants had different levels of visual difficulty and belonged to different age groups. Each interview lasted for around 10 minutes, with general and open-ended questions allowing users to express themselves better.
When asked about the need of help while using smartphone messaging, only 33.3% users claimed to be self-sufficient. Rest either required some or complete assistance.
A majority of participants were familiar with TalkBack/VoiceOver and used it frequently to access smartphones.
Despite being familiar with Braille, users did not prefer Braille keyboard.
Proposed solution
So it was clear that smartphones provided a lot of features even though the smooth screen posed a challenge.
Designing the layout
Designed a basic layout to include numbers and alphabets in a 3x3 grid layout.
We started exploring how we could arrange characters on each key of the 9-key layout. We initially proposed the following layout where the numbers and alphabets were accessible together in a button.
Developing the prototype
Developed the app in Flutter and pilot tested with Talkback.
I developed the application in Flutter to test it with the built-in accessibility feature in Android - TalkBack. The development involved a lot of logical processing along with some networking libraries to emulate a messaging application which worked in real time.
We conducted some pilot testing to see how the app was performing with TalkBack.
Testing the application
Tested the application with one of the previously interviewed participants.
After the basic application was ready, we tested the app with one of the users we had previously interviewed and received feedback on the issues.
Interaction with TalkBack made accessing the keyboard more difficult than easy. Despite pilot testing the integration, users faced challenges.
The keyboard provides limited characters and misses out on some important characters.
Redesigned the layout based on feedback and made it more compatible with Talkback.
Based on the insights and feedback received, we redesigned the layout to accommodate the needs of the users.
The following layout was designed and redeveloped.
We provided a button to switch between alphabets and numbers to provide ease of access for multiple characters.
On testing it with the participants again, they seemed to be happy with it and were able to use it.
The outcome
Developed a Flutter application with a 3x3 grid occupying half of the screen to make it easy for the VI users to locate characters.
The final app was developed keeping the user insights in mind and is ready for testing with the users.
Key features
The keyboard has a 3x3 grid of 9 keys, each associated with a set of characters.
The minimalistic layout and less number of keys reduce the cognitive load for the user and help them memorize the location of each character better.
The layout is analogous to the one in feature phones, making it easier for the VI users to use and ease the learning curve.
The application is designed in such a way that it works well with TalkBack/VoiceOver. Additionally, phonics are added to guide the user.
Impact
India HCI 2023
Publication
72.2%
Adoption rate
Of the 18 participants surveyed, 13 expressed an interest in using the 9-key layout as a preference over QWERTY keyboard.
The project deliverable was a research article which is published in India HCI 2023 conference.
Learnings and outcomes
Interacting with the target group
Conducting interviews with actual users helped the team understand and address the problems directly.Technical compatibility is as important as theoretical research
Using Flutter framework allowed testing the app with both TalkBack and VoiceOver, accommodating more users.Finding the target group for testing was a challenge
Due to the limited scope, we could not perform iterative testing.
To summarize
Designed and developed a 9-key-based keyboard layout compatible with Talkback for visually impaired users.
Even though technology is evolving rapidly, navigating smartphones remains inaccessible to visually impaired (VI) users. To address this issue, we explored several keyboard layouts and designed a 9-key-based keyboard layout to enhance the messaging experience. We conducted 18 interviews with VI users to understand their pain points and test the keyboard layout to find the optimal solution.
