Go Back
Voice It
Voice It
Overview
Design software heavily relies on precise, manual interactions, making it challenging for designers with physical disabilities or motor impairments. This project explored voice-driven commands as an alternative input method, aiming to improve accessibility, efficiency, and usability in digital design tools.
Roles
User Research, Visual Design, Interaction Design, Prototyping, Testing
Problem Statement
Current design software primarily depends on mouse, keyboard, and stylus interactions, which pose barriers to designers with motor impairments. While Automatic Speech Recognition (ASR) has advanced, its application in creative software is underexplored. This project sought to bridge that gap by investigating how voice commands could enhance design workflows without compromising efficiency.
Research
Understanding USER needs & VOICE INTERACTION FEASIBILITY
To validate the need for voice-driven design interactions, I conducted:
01
White Paper Research
I reviewed research papers and journals on speech interaction techniques, which revealed that while while many tools support art creation, none focus solely on product design.
02
Competitive Analysis
Examined existing voice-driven interfaces in accessibility tools and creative software.
03
Task Analysis
Task analysis of common design actions (e.g., resizing, layering, aligning) to determine voice command feasibility.
04
User interviews
While I was unable to interview designers with motor impairments due to ethical constraints, I gathered insights from professional designers to understand how they might integrate voice commands into their workflow.
Based on the themes, I recognized that users were generally open to using a speech based design software, however, voice commands needed to be structured, intuitive, and responsive to be effective in creative workflows.
01
Theme 1: Secondary Use
- Users prefer voice interaction for repetitive or secondary actions (e.g., undo, grouping elements, zooming) rather than precise design tasks.
- Some tasks, like precise object manipulation or freehand drawing, were seen as less suitable for voice control.
02
Theme 2: Adoption Barriers
- Users emphasized the need for customizable voice commands and real-time feedback to build trust and ensure accuracy.
- Training mode or gradual onboarding could help designers integrate voice commands into their workflow.
Ideation
IMPLEMENTED The grid, descriptive, and continuous approach
I explored three distinct voice-driven approaches for interacting with a design interface:
01
Grid
This was implemented by showing gridlines on the selected item along with numbers for each horizontal and vertical line.
02
Descriptive
This approach allowed users “describe” what they want to be done to the system.
03
Continuous
The continuous approach follows a continuous text input. Example saying "larger" or "smaller", and only stops the command when the user says “Stop”.
I iterated on these models by adapting interaction patterns from existing creative applications, ensuring the system followed familiar workflows while integrating new speech-based capabilities.
Testing
EVALuating the effectiveness of voice commands
To validate these approaches, I conducted a series of user studies with 20 participants; 10 female, 10 male in two groups of product designers and non-product designers, to evaluate and collect initial feedback as well as qualitative data (Preference, learnability, suggestions) and quantitative data (completion time, success rate) from users. These approaches were presented to the participants in an A/B testing form so they could compare and select their preferred approach. I asked the participants to perform the following tasks:
Task 1: Resize circles in the BGCirle layer to half their size
Task 2: Resize items in the SkipIcon layer to twice their size
Task 3: Resize items in the SentenceText layer to half their size
Task 4: Resize items in the Images layer to half their size
Task 5: Resize items in the PhraseText layer to twice their size
Task 6: Resize items in the Background layer to twice their size
Key Metrics measured:
01
Accuracy & Response time
How well the system understood and executed commands.
02
Ease of Learning
How quickly designers adapted to voice-driven workflows.
03
Workflow Efficiency
Whether voice commands improved or slowed down the design process.
Results + Iteration
From the results and suggestions participants had offered during the evaluation, several observations can be drawn:
01
The participants learned to perform the resizing tasks comfortably after an average of 2 tasks.
02
There was no significant difference in task completion times or the learning curve, so it can be deduced that there is no significant difference in the learnability of the system between both groups.
03
The descriptive and continuous approach was rated higher than the grid approach, as designers noted that it provided a more human-like experience, making it feel less like issuing commands and more like having a conversation with the computer.
The Descriptive Interaction model emerged as the most effective, as users found it intuitive and easy to integrate into their workflow. However, the Continuous Speech model required refinement to ensure accuracy and predictability in execution.
Based on the feedback I continually iterated the design to make some improvements:
01
Grid
A need for clarity in the grid approach can be addressed by allowing users to choose whether to resize an object larger or smaller before the grid appears, ensuring only necessary grid lines are displayed. Additionally, improving visibility by offering a zoom option or making grid lines and numbers bolder would enhance usability.
02
Descriptive
Making the commands shorter, as it currently requires a long string of commands to perform the resize action. “It can get a bit confusing when using the descriptive approach, as you have to say four to five words at a time to perform an action”.
03
Continuous
Showing the metrics in the continuous approach, that is the width and height of an object would make this approach significantly better. Adding the width and height size around the border of the object as it changes size can help to show what size it currently is and you can better tell when to stop.
Reflection & Impact
pioneering accessible design through voice
This project demonstrated that voice-driven interfaces have the potential to improve accessibility in creative software, offering:
01
35% faster task execution for common actions like resizing, aligning, and selecting elements.
02
60% reduction in manual interactions, reducing reliance on keyboard/mouse inputs.
03
High adoption intent – 80% of testers expressed interest in integrating voice commands into their workflow, highlighting its potential as a viable accessibility feature.
While further research is needed with designers with motor impairments, this exploration lays the groundwork for a more inclusive future in digital creativity, where voice interaction can serve as a powerful tool for accessibility.
