profile

Design Led

The Research Trap: Why most users give you misleading answers


After working with dozens of early-stage startups on their “new” product, as well as mentoring young designers in their product design skills, I’ve noticed one pattern:

Most designers, especially beginners, unknowingly use guiding questions that lead to false insights.

The Problem with Most UX Research

Here’s a typical user interview:

Designer: “Would you use a dashboard to track your metrics?”
User: “Yeah, that would be useful!”
Designer: “And would you want to see daily or weekly data?”
User: “Oh, daily sounds good!”
Designer: “Perfect!”

These are guiding questions, and they’re killing your research. Here’s why:

  1. They lead users to your solution
  2. They make it easy to say “yes” without commitment
  3. They tell you nothing about actual problems
  4. Users are polite - they’ll agree to sound helpful

Data point: In a study by Nielsen Norman Group, 82% of users said they would use a feature in research, but only 14% actually did when the feature was built. [Source: When to Use Which User Research Method]

The Psychology Behind Bad Research

Before we dive into solutions, let’s understand why users often mislead us (unintentionally):

Why People Say Yes But Don’t Use

There are three psychological principles at play:

  1. Social Desirability Bias Users want to be helpful and agreeable. When you show them a solution, they’ll likely say yes just to be nice. A study in the Journal of Consumer Research found that this bias can inflate positive responses by up to 40%. [Source: Response Bias and the Personality of Respondents]
  2. The Optimism Bias People consistently overestimate their future behavior. “Of course I’ll use a dashboard daily!” is similar to “Of course I’ll go to the gym every day!” Research by Tali Sharot shows we’re hardwired for this optimism. [Source: The Optimism Bias]
  3. The Feature Visualization Problem When users imagine a feature, they picture the perfect scenario: them having time, energy, and motivation to use it. Reality is messier.

Why Users Give Bad Solutions

There’s a famous Henry Ford quote: “If I had asked people what they wanted, they would have said faster horses.”

Here’s why users are bad at suggesting solutions:

  1. They’re experts in their problems, not solutions
  2. They can only suggest based on what they know exists
  3. They think in terms of current limitations

A Better Framework: Detective-Style Research

Instead of asking guiding questions, approach research like a detective investigating a case. Here’s what this looks like:

1. Observe First, Ask Later

Instead of: “Would you like better metric tracking?” Ask: “Show me how you made your last business decision.”

2. Look for Evidence

  • Open tabs in their browser
  • Notes on their desk
  • Tools they actually use
  • Workarounds they’ve created

3. Follow the Money

  • What are they currently paying for?
  • What problems cost them the most time?
  • Where do they lose revenue?

I prepared an exact interview script you can download following this framework, see below.

How to Uncover Real Pain Points

Most users struggle to articulate their real problems. This usually sounds something like this:

  • “We want better data insights.”
  • “Our plan is to streamline communication.”
  • “Oh yeah, we definitely want to improve there.”

What does that even mean, right?

Here’s why and how to fix it:

The Pain Articulation Problem

Users often:

  • Don’t want to seem incompetent
  • Have normalized their workarounds
  • Can’t identify root causes

Here’s a framework to dig deeper:

  1. The Status Quo Probe
    Instead of: “What’s your biggest problem?”
    Ask: “Walk me through your typical day”
    Then listen for sighs, pauses, or phrases like “it’s fine” or “we make it work”
  2. The Cost Investigation
    - “How long does this usually take?”
    - “What else could you do with that time?”
    - “What happens if this goes wrong?”
  3. The Emotion Hook
    Look for emotional responses:
    - Frustration (“This is so annoying”)
    - Resignation (“That’s just how it is”)
    - Workarounds (“We found a hack”)

A Mini Case Study

Recently, I watched a (pretty young) design team conduct research for a new analytics tool. Here’s what happened:

Their approach:

  • Asked if users wanted better analytics
  • Showed mockups of dashboards
  • Got positive feedback

The reality (discovered later):

  • Users checked one metric daily
  • Didn’t need real-time data
  • Wanted email alerts, not dashboards

Key learning: What users say they want ≠ What users will actually use

The Science Behind This

Recent research on user interviews shows:

  • Open-ended questions yield 5x more insights [Source: The Power of Open-Ended Questions]
  • Observational studies predict actual usage 3x better than direct questions
  • Past behavior predicts future behavior better than stated intentions

The Full-Stack Designers Toolbox:

Visual Design

Become 10x faster in designing interfaces by using a UI Kit. This makes sure your UI always looks good and you don’t always have to recreate the same components, like form inputs.

Keep in mind, using UI kits makes it even more important to think about a proper user journey and talk to the customers.

My favourite (free) UI Kit is Untitled UI.

Research

I created this free Interview Script to better uncover pain points. It’s the same script I use and comes with dozens of example questions. Here is the link to the Notion page (just duplicate it).

Business

Honestly, it’s super hard to convey the value of proper research. Startups want to get a full app design for $1.000 without paying for any research.

What worked for me in the past is to highlight the costs of “rebuilding the whole app”, which in 10/10 cases you have to do if you don’t do proper research, which can cost 20-30x more.

Tools

To record interviews, I use Fathom meeting recorder (which is free forever). The best thing are the meeting summaries you get, so you don’t have to take notes anymore.

Collaboration

Use links in Figma UI. Every feature should have a proper document (one of my clients use Notion for that), which you can link to in Figma, so that developers have some context.

Design Led

Every Sunday, you'll get a new lesson about product, design & startups to your inbox. Researched, heavily user focused & without fluff.

Share this page