XINGLU YAO
XINGLU YAO

Voyce_new

Senior Financial Scam is a severe social problem. Unfortunately, there has not been any effective solutions that directly speak to senior citizens’ needs and habits. In this project, I led the design of an AI system that empowers seniors to fight against financial scams.

I wish I could be there with her.

When we interviewed Brian, he wasn't in a good mood - a scammer who claimed to be an IRS official recently swindled his 80-year-old mom. "My mom lives alone. I had no idea what happened until she sent her money." Brian sighed, "It is not just the money she lost. It took her a long time to recover from her emotional turmoils and self-blames."

Brian's mom is not alone. Each year, elderly Americans lose as much as $37 billion to financial scams. In this project, I designed a voice based AI system that protects seniors from phone scams.

 
Type: Voice User Interface, Mobile Application
My Role: UX Design & User Research
Collaborators: Elsa Ho (User Research & PM), Kasturi Dani (Visual Design & Branding)
Sponsor: BlinkUX
 
 
 

User Research

7 weeks | 10 interviewees | 5 experts

Secondary Research: Read through reports from Federal Trade Commission, U.S. Department of Justice, Financial Fraud Research Center, and Bureau of Justice Statistics.

Subject Matter Experts: Interviewed five subject matter experts.

Field Visit and In-depth Interviews: Conducted field visits and in-depth interviews with five senior citizens and five family members of senior fraud victims.

 
 

Design Themes

independence | efficiency | accessibility | adaptability

 

INDEPENDENCE

Providing a sense of independence to seniors.   

 

EFFICIENCY

Delivering timely alerts to family members, without burdening them with excessive details.  

 
 
 

ACCESSIBILITY

Respecting seniors citizens’ routines, constraints and behavioral preferences.

 

ADAPTABILITY

Adapting to new types of phone scams quickly and effectively.

 
 
 

Ideation

6 steps | 55 ideas 

Step one: Ideated around different stakeholders (seniors, caregivers, banks, peers, communities, etc.)

Step two: Ideated around user journey (pre-scam, during-scam, after scam).

Step three: Ideated based on competitor’s products or related services.

Step four: "Wishing" - “If you have unlimited money, what service would you develop?”

Step five: "Worst Idea Ideation" - Generated "worst ideas" and turn them into good ones.

Step six: Affinity Diagramming - categorized all the ideas.

 
 

Evaluation and Selection

6 directions | choose AI System 

We narrowed down 55 ideas into six directions:  "scammer biometric characteristics identification", "AI assistant", "senior well-being monitoring", "bank accounts catered for seniors", "senior peer support and easy fraud report".

We identified “AI system” as the most promising direction to prototype

Narrowed down into six directions

Evaluated six directions.

 
 

Storyboards

4 scenarios

 
 

Flowchart

design user flows

 
 

Voice User Interface

designed for seniors

"Joyce" is the Artificial assistant that lives inside of "Voyce". Through a voice-based interaction, the artificial assistant Joyce cautions seniors and prompts seniors with next steps. We designed speech scripts based on the flowchart. Below is a few sample clips.

 
 

Mobile Interface

designed for senior's trusted family member

 

As for senior’s family members, the system provides an app to guide them through the on-boarding process, helps them customize settings, and sends notifications when seniors are having conversations with potential scammers or receiving suspicious voicemails.

The AI system gives each call a “ScamScore” to help family member judge if a phone call is fraudulent.

Onboarding Process:

Blocking Scam Calls:  

 
 

Mobile Interface UX Design

Information Architecture  | Wireframes

APP Sitemap

Wireframes

 
 

Testing Voice Interaction

user testing with seniors

We prototyped the AI assistant Joyce (Wizard of Oz prototyping) and went through main scenarios with seniors. By observing how seniors interacted with the audio prototypes, we learned how to improve the voice-based interaction system.
We also showed the system’s script and asked senior participants to revise it after interacting with the audio prototype. It helped us make the scripts more tailored to seniors specific needs.

Simplicity: Seniors expressed a desire for simplified conversation with the system. We the script to ask only yes or no questions. If seniors have to make a choice, Joyce will limit the options to two items.      

BEFORE                        
Joyce: “What do you want to do next?”
Senior: ...
AFTER                    
Joyce: “Do you want to end the conversation?”
Senior: "Yes."

Progressive disclosure: Seniors were overwhelmed by too many requests at one time. We modified the script to progressively guide the senior by presenting only one task at a time.                    

BEFORE
Joyce: “What can I do for you?”
Senior: “Add contact.”
Joyce: “Sure. Please provide the person's name and phone number.”
Senior: ...
AFTER
Joyce: “What can I do for you?”
Senior: “Add contact.”
Joyce: “Sure. What is the name of the person you would like to add?”
Senior: “Nancy.”
Joyce: "What is Nancy's phone number?"
Senior: "It's 111-111-111.
 
 

Testing Mobile Interface

user testing with seniors' family

BEFORE

AFTER

 
 
 

Stakeholder Map

the ecosystem behind Voyce

This chart gives a list of secondary stakeholders.

  • Voyce hosts its database on the cloud through cloud computing services.
  • Voyce collaborates with existing scammer blocking services to enrich its scam database.
  • Voyce works with Research Centers to update its fraud detection algorithms.
  • Voyce extracts public companies' information from seg.gov to evaluate if the call is coming from credible organizations.