Quantitative Fintech Surveys: Why I Miss the Market Research Interviewer

Lately, I’ve been lamenting the loss of the market research interviewer for quantitative surveys.

Online surveys are great—it’s an evolution that has provided many improvements in quantitative research. As online surveys replaced mall-intercept and telephone interviews, there was a dramatic reduction in costs and timing, and we were able to churn out more insights faster.  But, in our excitement to embrace these vital improvements, we hardly acknowledged a critical loss in the feedback loop: the contribution of an experienced interviewer.

An experienced interviewer would send up a yellow flag if they noticed respondents were confused by a concept or terminology in the questionnaire. Catching this early would allow me (or any researcher) to tweak the wording, or add additional prompts for clarification, before completing fieldwork. I have been thinking about this a lot as I survey consumers about their perceptions and adoption of various fintech products (e.g., digital payments, neobanks, crypto).

Fintech is a disruptive industry that is constantly evolving. With each new subcategory that emerges, new vocabulary is introduced, and the average consumer cannot keep up with the lexicon. Think of all the new terms associated with crypto products: blockchain, cryptocurrency, non-fungible tokens (NFTs), decentralized apps, etc.  It can be challenging to speak to consumers about these products, given various levels of understanding. As researchers, we know it is always important to write questionnaires in clear, concise, consumer-friendly language. Additional care must be taken when conducting market research on innovative or technical products and services, like fintech.

Fortunately, there are some practical steps we can take to make sure our questions are communicated effectively, with or without the assistance of an interviewer. I’ve identified three key areas for more effective FinTech surveys: Clearly Define Technical Terms, Draw from Brand or Culture and Build in Pre-tests to Fool-Proof.

Online Survey Best Practices for Fintech

1. Clearly Define Technical Terms

I highly recommend providing definitions of terms within the survey. Short definitions or clarifying statements can be incorporated directly into the question text. If longer definitions are required, they can be programmed into the survey as pop-up boxes that appear when the respondent hovers over certain words or phrases.

Before complex tasks like conjoint/discrete choice, we often recommend showing a glossary of key terms that can be reviewed again (and as often as necessary). The respondent simply clicks on a link on each subsequent screen to open the glossary again.

2. Draw from Brand or Culture

I have found that providing examples, like commonly known brands or products, can be very helpful. For example, when asking consumers about peer-to-peer (P2P) digital payments in the US, many are aware of Venmo or Zelle but are not familiar with the term “peer-to-peer digital payments”.  In a screener for P2P users, we might ask, “Have you used a digital app, such as Venmo, Cash App or Zelle, to send/receive money with friends and family?” Or, we could simply ask which products they have used to send/receive money to/from friends and family, providing a comprehensive list of P2P payment brands and traditional methods (e.g., cash, check).

3. Build in Pre-tests to Fool-Proof

What else can we do to integrate the value that an experienced interviewer brings? Organize a pre-test. Once the questionnaire and definitions have been drafted, invite a small group of relevant consumers to take the survey and take note of any areas of confusion. During the pretest, additional questions can be asked to collect feedback on the survey itself. These questions would prompt respondents to call out any confusing questions or terminology. If budget allows, a moderated pretest can be especially effective at identifying confusing language. In a moderated pretest, participants complete the online survey while a researcher observes and encourages them to discuss their thought process as they read and answer each question. The researcher generally avoids providing any direction or clarification, waiting until the respondent has completed all survey questions to fully discuss areas of concern.

Although this may require a little extra time and budget at the beginning of a project, conducting pretests can be well worth the effort for novel categories, like fintech, or new question types. If an official pretest is just not feasible, ask colleagues who are less familiar with the subject matter to review the survey and provide constructive feedback.

The most important takeaway here is to ensure you are set up to capture pertinent responses that effectively answer your overarching business questions. The quality of the insights gained from quantitative surveys is highly dependent on the quality and clarity of the questions asked. Even if we must ask them without highly experienced interviewer.