Why “Describe the Last Time You Did This” Can Fail in User Interviews?
Tell me about the last time…’ isn't a magic interview question. The quality of the answer depends on the context, comfort, and restraint of the interviewer.
Why user interviews?
Interviews are what I would call a goldmine for getting insights into your product. You can be a novice and come out of the interview as an expert on user behaviour. When you interview users, you uncover insights that are usually not visible in data or random research done through AI. The only other source that I think comes close to User Interviews is Support Tickets or Escalation calls, where you see actual frustration of users using your Product.
The Problem
Conducting user interviews seems easy until you do it on your own. It actually does not matter whether you are an experienced Product manager or just starting out. When you are working on an unknown problem, you will tend to be biased towards a solution. Not even an experienced product manager can escape this default behaviour.
I have conducted 100s of interviews; I do not think there has ever been one that I felt really proud of. It always felt like: Why did I direct the user to a specific answer? Was there a bias in my questioning…? and so on.
Another problem with customer discovery interviews is asking the very question that is considered the nail in the coffin: Can you describe the last time you did this? On the surface, this question seems simple, but it can push the user out of their comfort zone. As soon as you ask this question, the user can become self-aware of the action they took. Sometimes it sounds too personal or poking. The major reason is that as soon as you ask this question, the user tends to become self-aware of what they did. Some decisions they might even feel embarrassed to talk about publicly, like choosing a cheaper option. Such behaviour tends to defeat the whole purpose of the interview itself.
One of the best bets is to make the user as comfortable as possible. Interviewers like me tend to jump to the main point of discussion without being able to make the user feel welcome. In fact, this is one of the skills that Product Managers should possess - the ability to make small talk. Once the user is comfortable, you still cannot go on asking direct questions to describe their last experience. What I would do is try to be more exploratory in my approach. For example, if I’m interviewing a user of a Running app, I would talk a lot about their interest, if they participate in formal events, what makes them happy, and when I feel that they are in the Zone, it is at that point I would, in the most natural way, ask what the last time they went for a run?
In the search for the perfect storyline, you will end up polluting the plot.
You have not perfected the interview yet by being natural. It is when you learn to hold yourself from interrupting the user in the middle of what they are talking. In the search for the perfect storyline, you will end up polluting the plot. As soon as the user says, Well, the last time, ummm… I went to the park and started jogging; your inner instinct will call you to ask- so you did not start tracking? It is these moments of impulse that, as a product manager, you need to be aware of - learning to be patient to listen to the whole story before judging or worse manipulating the narrative.
Write your questions on a piece of paper to be asked later if not discovered naturally. A simple activity will take you miles :P while interviewing the running app user.
I’m working on hands-on tools to help readers practise product management and not just learn through theory. If you find these helpful, try out the Customer Interview tool at www.productplayground.io