Assistivity

Assistivity

Assistivity

Duration

Jan 2025 - Present

Stakeholders

ASSETS 2025,
Dr. JooYoung Seo

Team

Kenneth Ge, Cofounder
Priscilla Zhang, Cofounder

My Role

I led across all aspects of the Assistivity project, from interface design and development to preliminary research. I collaborated with Kenny and Priscilla on conducting user research, creating prototypes, and co-authoring a paper. I led team design sessions to brainstorm and iterate on interface ideas, helped implement the voice-powered workspace, and supported refining the user experience based on early feedback.

Skills

UI/UX Design
UX Research
Front-End Development
Voice Interface Design
Prototyping & Iteration
Usability Testing
Qualitative Research
Cross-Functional Collaboration
Design Facilitation

Tools

Figma
Svelte
JavaScript
HTML/CSS
Zoom Meeting
Google Suite
Github
Overleaf

Duration

Jan 2025 - Present

Stakeholders

ASSETS 2025,
Dr. JooYoung Seo

Team

Kenneth Ge, Cofounder
Priscilla Zhang, Cofounder

My Role

I led across all aspects of the Assistivity project, from interface design and development to preliminary research. I collaborated with Kenny and Priscilla on conducting user research, creating prototypes, and co-authoring a paper. I led team design sessions to brainstorm and iterate on interface ideas, helped implement the voice-powered workspace, and supported refining the user experience based on early feedback.

Skills

UI/UX Design
UX Research
Front-End Development
Voice Interface Design
Prototyping & Iteration
Usability Testing
Qualitative Research
Cross-Functional Collaboration
Design Facilitation

Tools

Figma
Svelte
JavaScript
HTML/CSS
Zoom Meeting
Google Suite
Github
Overleaf

Duration

Jan 2025 - Present

Stakeholders

ASSETS 2025,
Dr. JooYoung Seo

Team

Kenneth Ge, Cofounder
Priscilla Zhang, Cofounder

My Role

I led across all aspects of the Assistivity project, from interface design and development to preliminary research. I collaborated with Kenny and Priscilla on conducting user research, creating prototypes, and co-authoring a paper. I led team design sessions to brainstorm and iterate on interface ideas, helped implement the voice-powered workspace, and supported refining the user experience based on early feedback.

Skills

UI/UX Design
UX Research
Front-End Development
Voice Interface Design
Prototyping & Iteration
Usability Testing
Qualitative Research
Cross-Functional Collaboration
Design Facilitation

Tools

Figma
Svelte
JavaScript
HTML/CSS
Zoom Meeting
Google Suite
Github
Overleaf

ASSETS 2025 POSTERS AND DEMONSTRATIONS

Kenneth Ge, Ryan Paul, Priscilla Zhang, JooYoung Seo

Kenneth Ge, Ryan Paul, Priscilla Zhang, JooYoung Seo

Work in Progress

This case study is still under development. The content is currently in a draft form, with storytelling, visuals, and full context to be added soon.

Work in Progress

This case study is still under development. The content is currently in a draft form, with storytelling, visuals, and full context to be added soon.

Work in Progress

This case study is still under development. The content is currently in a draft form, with storytelling, visuals, and full context to be added soon.

Overview

Overview

Overview

About

DESCRIPTION

We created a voice powered math workspace designed for individuals with fine-motor disabilities. The workspace allows direct dictation of math symbols and provides a visual network of connected math relationships.

Product Vision

DESCRIPTION

We want to make math accessible for individuals with fine-motor disabilities and provide a new medium for everyone to engage with math. We believe a voice engagement with math will inspire a deeper understanding of mathematics, which focuses less on input mechanics and more on conceptual understanding.

About

DESCRIPTION

We created a voice powered math workspace designed for individuals with fine-motor disabilities. The workspace allows direct dictation of math symbols and provides a visual network of connected math relationships.

Product Vision

DESCRIPTION

We want to make math accessible for individuals with fine-motor disabilities and provide a new medium for everyone to engage with math. We believe a voice engagement with math will inspire a deeper understanding of mathematics, which focuses less on input mechanics and more on conceptual understanding.

About

DESCRIPTION

We created a voice powered math workspace designed for individuals with fine-motor disabilities. The workspace allows direct dictation of math symbols and provides a visual network of connected math relationships.

Product Vision

DESCRIPTION

We want to make math accessible for individuals with fine-motor disabilities and provide a new medium for everyone to engage with math. We believe a voice engagement with math will inspire a deeper understanding of mathematics, which focuses less on input mechanics and more on conceptual understanding.

Solution

A voice-powered math tool designed to make problem-solving accessible for users with limited mobility.

Solution

A voice-powered math tool designed to make problem-solving accessible for users with limited mobility.

Solution

A voice-powered math tool designed to make problem-solving accessible for users with limited mobility.

Context

Context

Context

Problem

DESCRIPTION

There is a large underserved market for assistive technologies in mathematics for people with mobility disabilities. Additionally, existing tools are often rigid and difficult for students to integrate into their existing workflows.

Opportunity

DESCRIPTION

Recent advancements in AI open the door to creating seamless voice-based math tools that better support individuals with mobility disabilities. Unlike existing rigid solutions, this approach can fit naturally into students’ workflows, removing friction and reliance on traditional interfaces. The opportunity lies in designing a tool that not only works intelligently but also feels intuitive and empowering to use.

Problem

DESCRIPTION

There is a large underserved market for assistive technologies in mathematics for people with mobility disabilities. Additionally, existing tools are often rigid and difficult for students to integrate into their existing workflows.

Opportunity

DESCRIPTION

Recent advancements in AI open the door to creating seamless voice-based math tools that better support individuals with mobility disabilities. Unlike existing rigid solutions, this approach can fit naturally into students’ workflows, removing friction and reliance on traditional interfaces. The opportunity lies in designing a tool that not only works intelligently but also feels intuitive and empowering to use.

Problem

DESCRIPTION

There is a large underserved market for assistive technologies in mathematics for people with mobility disabilities. Additionally, existing tools are often rigid and difficult for students to integrate into their existing workflows.

Opportunity

DESCRIPTION

Recent advancements in AI open the door to creating seamless voice-based math tools that better support individuals with mobility disabilities. Unlike existing rigid solutions, this approach can fit naturally into students’ workflows, removing friction and reliance on traditional interfaces. The opportunity lies in designing a tool that not only works intelligently but also feels intuitive and empowering to use.

Challenge

How might we empower students with fine motor disabilities to express their Math knowledge without physical limitations?

Challenge

How might we empower students with fine motor disabilities to express their Math knowledge without physical limitations?

Challenge

How might we empower students with fine motor disabilities to express their Math knowledge without physical limitations?

Research

Research

Research

Framing the Problem

DESCRIPTION

To better understand the target audience and their environment, I reviewed both academic and non-academic literature to fill in initial knowledge gaps. I focused not only on the immediate challenges of math completion but also on the broader experiences of people with fine motor disabilities.

Framing the Problem

DESCRIPTION

To better understand the target audience and their environment, I reviewed both academic and non-academic literature to fill in initial knowledge gaps. I focused not only on the immediate challenges of math completion but also on the broader experiences of people with fine motor disabilities.

Framing the Problem

DESCRIPTION

To better understand the target audience and their environment, I reviewed both academic and non-academic literature to fill in initial knowledge gaps. I focused not only on the immediate challenges of math completion but also on the broader experiences of people with fine motor disabilities.

Precision, Control, and Cognitive Load

Math relies heavily on fine motor precision, writing formulas, drawing graphs, and using symbolic notation. These tasks can be physically inaccessible and time-consuming, often shifting focus away from core learning goals. Unfortunately, traditional classroom tools and environments rarely account for these barriers.

Identity and Social Pressure

Students hesitate to seek accommodations due to stigma, a desire to appear “normal,” or fear of being judged. For those with invisible disabilities, the pressure to “prove” their condition often leads to non-disclosure and disengagement.

Tool Overload and Mistrust

Many students enter college unaware of what disability services exist or how to access them. Navigating new tools and systems, especially without guidance, can feel overwhelming. Past negative experiences with faculty or unhelpful services cause some to question the value of available support.

Understaffed and Under-Resourced

Disability service offices often operate with limited staff and resources, leaving little room for personalized outreach. Tools like assistive tech and mentorship are valued but frequently blocked by funding issues or institutional red tape.

Policy vs. Practice

Though policies exist, students still encounter practical barriers to accessing accommodations. Services often follow rigid protocols instead of adapting to individual needs. A student’s cultural background, disability type, and past school experiences all influence whether they feel empowered or hesitant to engage with these services. While students value autonomy and self-advocacy, they also need knowledgeable support to navigate the system.

Literature Review + Empathy Map

DESCRIPTION

I continued exploring literature on fine motor disabilities and created an empathy map to deepen my awareness of the user experience. I also collaborated closely with a teammate who experienced severe wrist pain, an issue that ultimately led them to withdraw from their math course. Our weekly conversations, paired with insights from the literature, strongly shaped my early design decisions. Link for full empathy map.

Literature Review + Empathy Map

DESCRIPTION

I continued exploring literature on fine motor disabilities and created an empathy map to deepen my awareness of the user experience. I also collaborated closely with a teammate who experienced severe wrist pain, an issue that ultimately led them to withdraw from their math course. Our weekly conversations, paired with insights from the literature, strongly shaped my early design decisions. Link for full empathy map.

Literature Review + Empathy Map

DESCRIPTION

I continued exploring literature on fine motor disabilities and created an empathy map to deepen my awareness of the user experience. I also collaborated closely with a teammate who experienced severe wrist pain, an issue that ultimately led them to withdraw from their math course. Our weekly conversations, paired with insights from the literature, strongly shaped my early design decisions. Link for full empathy map.

Asset A
Asset A
Asset A
Asset A

Persona

Our persona was inspired by conversations with a team member who experiences challenges related to fine motor disabilities, as well as by insights from our literature review.

Persona

Our persona was inspired by conversations with a team member who experiences challenges related to fine motor disabilities, as well as by insights from our literature review.

Persona

Our persona was inspired by conversations with a team member who experiences challenges related to fine motor disabilities, as well as by insights from our literature review.

Asset A

Bio

Alex is a 20-year-old B.Sc. Mathematics (Honours) student in his second year at the University of California, Santa Cruz, where he lives on campus. Living with Carpal Tunnel Syndrome, he faces daily challenges writing equations by hand, taking notes, and completing assignments efficiently. He actively searches for voice dictation tools to support his STEM coursework but often finds existing options ineffective in understanding math symbols or poorly suited to his coursework.

Goals

  • Successfully complete homework without feeling exhausted and mentally drained

  • Document his mathematical thinking and solutions clearly and independently

  • Maintain academic integrity without feeling like he is at a disadvantage

Pains

  • Struggles with writing complex equations manually for extended periods, leading to slow and inefficient work

  • Existing tools are ill-suited for STEM notation, and relying on peers for note-taking often results in misunderstandings

  • Experiences exhaustion and discomfort from prolonged writing sessions, fearing it may cause him to fall behind in coursework

Workarounds

  • Arrives early to class to discuss accommodations with instructors

  • Relies on a combination of typing, speech-to-text, and note-sharing with peers

  • Uses assistive technology when available but finds them lacking for STEM-specific tasks

  • Participates actively in study groups to verbally process concepts and reduce writing workload

Competitive Analysis

DESCRIPTION

It was important to understand who the competitors were and what they were working on. Reviewing the literature revealed both existing solutions and emerging generative AI tools entering the market.

Competitive Analysis

DESCRIPTION

It was important to understand who the competitors were and what they were working on. Reviewing the literature revealed both existing solutions and emerging generative AI tools entering the market.

Competitive Analysis

DESCRIPTION

It was important to understand who the competitors were and what they were working on. Reviewing the literature revealed both existing solutions and emerging generative AI tools entering the market.

Loading...

Most speech-based math editors for Fine motor disability users rely on rigid grammar and structured input, which make them difficult to use in classroom settings and inaccessible in real-world environments despite showing potential for improving accessibility.

Most speech-based math editors for Fine motor disability users rely on rigid grammar and structured input, which make them difficult to use in classroom settings and inaccessible in real-world environments despite showing potential for improving accessibility.

Most speech-based math editors for Fine motor disability users rely on rigid grammar and structured input, which make them difficult to use in classroom settings and inaccessible in real-world environments despite showing potential for improving accessibility.

Bio

Alex is a 20-year-old B.Sc. Mathematics (Honours) student in his second year at the University of California, Santa Cruz, where he lives on campus. Living with Carpal Tunnel Syndrome, he faces daily challenges writing equations by hand, taking notes, and completing assignments efficiently. He actively searches for voice dictation tools to support his STEM coursework but often finds existing options ineffective in understanding math symbols or poorly suited to his coursework.

Goals

  • Successfully complete homework without feeling exhausted and mentally drained

  • Document his mathematical thinking and solutions clearly and independently

  • Maintain academic integrity without feeling like he is at a disadvantage

Pains

  • Struggles with writing complex equations manually for extended periods, leading to slow and inefficient work

  • Existing tools are ill-suited for STEM notation, and relying on peers for note-taking often results in misunderstandings

  • Experiences exhaustion and discomfort from prolonged writing sessions, fearing it may cause him to fall behind in coursework

Workarounds

  • Arrives early to class to discuss accommodations with instructors

  • Relies on a combination of typing, speech-to-text, and note-sharing with peers

  • Uses assistive technology when available but finds them lacking for STEM-specific tasks

  • Participates actively in study groups to verbally process concepts and reduce writing workload

MathGPT, Mathful and StudyFetch

DESCRIPTION

AI powered tools had more contextual understanding, however, these tools are designed to act as tutoring bots, focused on delivering explanations, rather than supporting students in authoring or expressing math.. As a result, they serve a different purpose and do not address the accessibility needs of students with fine motor disabilities.

EquatIO

DESCRIPTION

Among the tools reviewed, EquatIO stood out for its voice input support and seamless integration as a browser plugin, particularly with platforms like Google Docs. However, it still struggled with contextual understanding, often producing incomplete or inaccurate translations. Due to the limited user data available on its capabilities, we decided to test it ourselves.

MathGPT, Mathful and StudyFetch

DESCRIPTION

AI powered tools had more contextual understanding, however, these tools are designed to act as tutoring bots, focused on delivering explanations, rather than supporting students in authoring or expressing math.. As a result, they serve a different purpose and do not address the accessibility needs of students with fine motor disabilities.

EquatIO

DESCRIPTION

Among the tools reviewed, EquatIO stood out for its voice input support and seamless integration as a browser plugin, particularly with platforms like Google Docs. However, it still struggled with contextual understanding, often producing incomplete or inaccurate translations. Due to the limited user data available on its capabilities, we decided to test it ourselves.

MathGPT, Mathful and StudyFetch

DESCRIPTION

AI powered tools had more contextual understanding, however, these tools are designed to act as tutoring bots, focused on delivering explanations, rather than supporting students in authoring or expressing math.. As a result, they serve a different purpose and do not address the accessibility needs of students with fine motor disabilities.

EquatIO

DESCRIPTION

Among the tools reviewed, EquatIO stood out for its voice input support and seamless integration as a browser plugin, particularly with platforms like Google Docs. However, it still struggled with contextual understanding, often producing incomplete or inaccurate translations. Due to the limited user data available on its capabilities, we decided to test it ourselves.

Review

DESCRIPTION

We tested EquatIO with three spoken math prompts. While it partially recognized the structure of integrals and equations, it consistently struggled with limits, notation, and variable mapping. Key issues included misinterpreting phrases like “pi over two” and failing to associate terms like “index of refraction” with variables (e.g., n). Overall, the tool produced incomplete or inaccurate outputs. Link for full breakdown.

Review

DESCRIPTION

We tested EquatIO with three spoken math prompts. While it partially recognized the structure of integrals and equations, it consistently struggled with limits, notation, and variable mapping. Key issues included misinterpreting phrases like “pi over two” and failing to associate terms like “index of refraction” with variables (e.g., n). Overall, the tool produced incomplete or inaccurate outputs. Link for full breakdown.

Review

DESCRIPTION

We tested EquatIO with three spoken math prompts. While it partially recognized the structure of integrals and equations, it consistently struggled with limits, notation, and variable mapping. Key issues included misinterpreting phrases like “pi over two” and failing to associate terms like “index of refraction” with variables (e.g., n). Overall, the tool produced incomplete or inaccurate outputs. Link for full breakdown.

Asset A

Takeaways

We used an affinity diagram to synthesize themes from our literature review and conversations with our team member. These themes guided our feature development.

Takeaways

We used an affinity diagram to synthesize themes from our literature review and conversations with our team member. These themes guided our feature development.

Takeaways

We used an affinity diagram to synthesize themes from our literature review and conversations with our team member. These themes guided our feature development.

Robust Voice Interpretation

Existing tools often require users to follow rigid voice commands, disrupting focus and slowing problem-solving. Frequent transcription errors can break the flow of thinking and make it harder to engage with complex concepts. Assistivity should be designed to interpret natural variations in speech, making voice input smoother and more intuitive.

  • Recognizes a wide range of spoken math expressions, symbols, and notation

  • Supports natural speech patterns, no need to memorize strict commands

  • Reduces friction in problem-solving by improving transcription accuracy

Flexible Editing & Exporting

Many tools make it hard to revise or reuse math once it's been transcribed. Complex commands and clunky interfaces get in the way of quick edits or exporting work. Assistivity should enable flexible editing through voice and simple interactions, making it easier to refine ideas or share equations across platforms like Word or Docs.

  • Edit equations with voice or minimal interaction, no manual rework

  • Easily export math to tools like Microsoft Word

  • Supports iteration and sharing throughout the problem-solving process

Clear Logical Structure

Mathematical thinking is rarely linear. Solving complex problems often means breaking them down into smaller, connected parts. Tools that flatten this process can make it harder to follow or explain solutions. Assistivity should helps users map relationships between steps clearly, supporting both understanding and communication.

  • Reflects how people actually solve multi-step math problems

  • Visually organizes related expressions and steps

  • Makes complex solutions easier to read, review, and refine

Supports Diverse Problem-Solving

Styles

Supports Diverse Problem-Solving Styles

Supports Diverse

Problem-Solving Styles

Problem-solving isn’t one-size-fits-all, some users speak through problems, others jump between steps or revise along the way. Rigid tools can interrupt these natural workflows. Assistivity should be designed to adapt to individual styles, offering structure where needed and flexibility where it matters.

  • Personalize how equations are entered, edited, and reviewed

  • Supports non-linear problem-solving and exploration

  • Adapts to different thinking styles without forcing a fixed process

Loading...

Features

Features

Features

Robust Voice Interpretation

DESCRIPTION

A voice engine that handles STEM vocabulary, well-known equations, and notation, enabling commands like “change plus to minus” or “substitute x with 7” while filtering out non-math speech.

Robust Voice Interpretation

DESCRIPTION

A voice engine that handles STEM vocabulary, well-known equations, and notation, enabling commands like “change plus to minus” or “substitute x with 7” while filtering out non-math speech.

Robust Voice Interpretation

DESCRIPTION

A voice engine that handles STEM vocabulary, well-known equations, and notation, enabling commands like “change plus to minus” or “substitute x with 7” while filtering out non-math speech.

Asset A
Asset A

Flexible Editing & Exporting

DESCRIPTION

Each equation set exists in its own node, with tools for printing, copying (Word/LaTeX), adding connected subproblems, and arranging equations. Supports voice, WYSIWYG, and LaTeX editing.

Flexible Editing & Exporting

DESCRIPTION

Each equation set exists in its own node, with tools for printing, copying (Word/LaTeX), adding connected subproblems, and arranging equations. Supports voice, WYSIWYG, and LaTeX editing.

Flexible Editing & Exporting

DESCRIPTION

Each equation set exists in its own node, with tools for printing, copying (Word/LaTeX), adding connected subproblems, and arranging equations. Supports voice, WYSIWYG, and LaTeX editing.

Asset A
Asset A

Clear Logical Structure

DESCRIPTION

A visual map of connected equation nodes using color saturation to show hierarchy, enabling modular, branching problem solving. Animated previews reduce clutter.

Clear Logical Structure

DESCRIPTION

A visual map of connected equation nodes using color saturation to show hierarchy, enabling modular, branching problem solving. Animated previews reduce clutter.

Clear Logical Structure

DESCRIPTION

A visual map of connected equation nodes using color saturation to show hierarchy, enabling modular, branching problem solving. Animated previews reduce clutter.

Asset A
Asset A

Supports Diverse Problem-Solving Styles

DESCRIPTION

Supports uploads, pen markup in multiple styles, and saving/loading work to encourage creative, multi-step problem solving beyond transcription.

Supports Diverse Problem-Solving Styles

DESCRIPTION

Supports uploads, pen markup in multiple styles, and saving/loading work to encourage creative, multi-step problem solving beyond transcription.

Supports Diverse Problem-Solving Styles

DESCRIPTION

Supports uploads, pen markup in multiple styles, and saving/loading work to encourage creative, multi-step problem solving beyond transcription.

Asset A
Asset A

Putting It to the Test

DESCRIPTION

The project comes full circle with a final test. Using the same challenge that had previously exposed competitors’ shortcomings, we put our tool to the test. The outcome? A clear, accurate, and user-friendly result, showcasing how far the solution has come since the first iteration.

Putting It to the Test

DESCRIPTION

The project comes full circle with a final test. Using the same challenge that had previously exposed competitors’ shortcomings, we put our tool to the test. The outcome? A clear, accurate, and user-friendly result, showcasing how far the solution has come since the first iteration.

Putting It to the Test

DESCRIPTION

The project comes full circle with a final test. Using the same challenge that had previously exposed competitors’ shortcomings, we put our tool to the test. The outcome? A clear, accurate, and user-friendly result, showcasing how far the solution has come since the first iteration.

Asset A
Asset A
Asset A

Future Direction

Future Direction

Future Direction

Sneak Peek

A look at some of the designs in progress. More to come soon…

Sneak Peek

A look at some of the designs in progress. More to come soon…

Asset A
Asset A
Asset A
Asset A

Sneak Peek

A look at some of the designs in progress. More to come soon…

Sneak Peek

A look at some of the designs in progress. More to come soon…

Asset A
Asset A
Asset A

Enter Password