What if relaxation technology could adapt to how you feel?
Existing sound wellness products rely on fixed libraries of pre-recorded soundscapes
What if: we could personalize relaxation by adapting sound to a person's emotional state?
I built Mist,
an AI powered and emotion aware sound engine
Stage 1: User Input
The user describes how they are feeling to mist, in natural language
Stage 2: LLM Extraction & soundscape composition
Mist uses AI to map those emotions into researched psychoacoustic parameters
Artificial Intelligence?
How does the product backend work?
1
I built a 2 stage AI pipeline that breaks down a users natural input into musical KPIs
2
These KPIs guide recipe formation of the soundscape from a library of pre-curated sound library
Mist does not generate any audio using AI,
as that would be non-deterministic
The Problem
What v1 got wrong:
Thin user input = poor personalization
Whoops.
How did I discover this?
The insight surfaced when testing with the first 8 users
the 8 soundscape recipes showed 43% similarity in recipe,
and utilized only 13% of all audio components in the sound library!
Why was this happening?
To discover why the AI pipeline was failing,
I analyzed the data from the 8 soundscapes
What the data revealed
Many parameters had the same values across users,
because the AI couldn’t find a clear signal in the user’s input and used a default value.
My initial assumption
user’s descriptions of their state of mind would contain the emotional signals required
What user testing revealed
unstructured + natural input often lacked the signals needed to score all four parameters
/ phrasing the problem
how might we
bridge the gap between what users say and what the system needs?
Idea one:
Effort reducers to guide the users input, & help them spend less time thinking
1
Emotional Cues:
Select how you feel
2
Common Situations:
Choose a frame of mind
3
Imagery Prompts:
Choose an atmosphere
Tradeoffs in this idea:
recognition (that’s me) is a lower bar for entry than generation (this is what I am thinking)
doesn’t rely on raw insights from the user, which is the MVP and differentiator
Idea two:
Breaking the intake flow into multiple questions targeting specific parameters
1
Intake start:
Open ended prompt targeting valence, the broadest schema.
2
Fixed follow ups:
Pre-written follow up questions that target each schema
Tradeoffs in this idea:
detailed information about each schema = more accurate guess
highly inconvenient for the user to
answer multiple questions
Idea three:
Generative follow-up questions
Conditional Follow Ups:
Targeted follow ups with generated questions if the system cannot detect few schemas
Tradeoffs in this idea:
generative copywriting can be personalized
to situation
every schema does not benefit from
descriptive input
The Final Intake Flow
Combining best practices:
a generative flow that combines open-ended & close-ended follow ups
Impact after changes
Repeating the same prompts with 8 test users
resulted in significantly more diverse and personalized soundscapes.
across 8 users
before
after
recipe similarity
43 %
21 %
library utilization
13 %
56 %
Learnings & Conclusions
This was but a snapshot of my process, let’s connect to take a deeper dive!
Designing systems and behaviors, not screens
Learning to design the relationships, states, and behaviors that make an experience work as a system
Bringing faster time to activation by understanding technical constraints
Understanding them can reveal opportunities to remove friction and shorten the path to action.