Guiding better search inputs with examples
Little Context about the service
When people face a legal issue, they search for answers before going to a lawyer. But existing systems rely on keywords and filters that require legal expertise that most people don't have. We solved this by using LLM, where people describe their situation as-is, and AI finds the most relevant cases.
ROLE & Contribution
Product Designer. Identified the gap between users' mental model and effective LLM search input, then designed and validated a sample-based guidance system from concept to production.
Problem
Although we encouraged free-form input, most users defaulted to keywords or compressed their situation into a single phrase.
The problem was that when users provide too little context, retrieval quality suffers. And when the results feel generic or unhelpful, users don't think ‘I should've given more detail’, and it ultimately led to dissatisfaction with the product.
Limited context
AI retrieves broad precedents
Results feel irrelevant
Dissatisfaction
Problem Evidence
To better understand this pattern, we analyzed search queries from the closed beta and compared user satisfaction based on the level of query detail.
We found that users who submitted compressed search inputs reported 60% satisfaction, compared with 85% for users who provided detailed inputs.
Why Compressed inputs?
Even though the interface explicitly asked users to "tell us about your situation as detailed as possible"...
... when we followed up with users who submitted compressed inputs, we found that each had a different understanding of what "detailed" meant. In other words, users had no clear criteria for what counted as "detailed," only with an abstract UI prompt.
"I figured this much would be enough to get what I was looking for.”
Detailed = this much
“I wasn't sure how far I was supposed to go with the details.”
Detailed = ?
“Just made my keywords more specific. That's how I'm used to searching”
Detailed = Specific keywords
Approach
To make abstract guidance concrete, we introduced sample situations that users could use as a reference when describing their own cases.
Design Principles & Explorations
1. Design samples users would actually refer to
Writing sample
There's a guy who owns a small business, and not long ago, things were rough because of stiff competition and a bad economy. He could barely pay his employees or keep up with loan payments. Instead of sorting things out, he started using company money for personal expenses like vacations and car costs, claiming it was for "business purposes." Our business quickly spiraled into deeper financial trouble, leaving us on the brink of bankruptcy.
Generic sample
Feels disconnected from users’ situations
→ Easy to ignore
Category-specific sample
Selected
Encourages exploration
Relevant to users’ own situations
→ Easier to model their own input after
2. Make samples quickly scannable
Writing sample
Random Topic
Criminal Offenses
Civil Matters
Family Law
Labor and Employment
Real Estate
Traffic Violations
Intellectual Property
Consumer Issues
Financial and Investment
Medical Malpractice
Sample 1
There's a guy who owns a small business, and not long ago, things were rough because of stiff competition and a bad economy. He could barely pay his employees last year or keep up with loan payments. Instead of sorting things out, he ...
Sample 2
I was walking along Main Street in the evening, minding my own business as usual. Suddenly, I noticed a group of people approaching from my left, coming from the side of a nearby club. Among them was a man who looked familiar. ...
Multiple sample at a time
Competes for users’ attention
Combination of Tags+multiple samples
→ Increases cognitive load before writing
One representative sample
Selected
One clear focal point
Faster to understand the expected structure
→ Easier to start writing immediately
10 Categories + 10 Subcategories
More precise sample matching
Finding the perfect sample type requires extra effort
→ Slower to start reading
10 Categories
Selected
Relevant enough to be useful
→ Enables immediate access to examples
Dropdown
Options hidden until opened
Requires an extra click to explore
→ Slower to switch between topics
Category Tags
Selected
All options visible at a glance
One-click exploration
→ Faster to browse different examples
Validation
We hypothesized that concrete examples would encourage more specific inputs than having no reference, and that relatable examples would further amplify this effect. To validate this, we had 4 users input their situation under three conditions.
This confirmed that specificity, not just the presence of a sample, drives richer user input. And we prioritized category-specific samples in the final design.
Solution
Learnings
Friction in how samples are shown mattered as much as the content.
Good sample content wasn't enough on its own. Whether it was choosing one sample over many, simplifying categories, or replacing dropdowns with tags, each decision came down to reducing the steps between seeing a sample and using it.
Relatability goes beyond a small detail.
We assumed any sample would encourage better input. Instead, users only wrote more detailed responses when the sample matched their own legal situation. The difference was whether users could see themselves in it.
Convenience in search doesn't guarantee reliable results.
Even with a comfortable search experience by LLM, getting users to reliable outcomes is a separate area, and figuring out what conditions make that possible is something we'll need to keep digging into.