My Recap of Analytics & Insights Summit

Inspiring. Insightful. Challenging. Thought-provoking. Daunting. Encouraging. These are just some of the words I’ve heard spoken by others to encapsulate the experience of attending this year’s Analytics & Insights Summit hosted by Sawtooth Software. This annual conference is, in my opinion, the pinnacle event for analytics that is accessible to practitioners in the market research industry. Unlike many conferences with large ballrooms of company-sponsored booths hawking their wares, there is no selling. Instead, there are academics and practitioners mingling and sharing ideas, collaborating, challenging one another, and striving for advancement. Presentations are focused on methodology, not proprietary software. And gratefully, TRC Insights realizes this aspect by ensuring representation (aka, me) at an event so vital to our livelihood and the capabilities we bring to our clients.

And in the spirit of sharing, I’d like to share three broad takeaways from the conference: (1) artificial intelligence (AI) in market research, (2) data quality and user experience (UX), and (3) important but invisible improvements to current methodologies.

#1: AI in Market Research

I suppose in today’s current environment I cannot ignore AI so let’s tackle this one first. There were nearly a dozen talks where AI was either the primary or secondary topic. And where AI was primary, the authors directly tackled the issue of whether AI could be reliably used to elicit insights for business decision making. Experiments were conducted and tested through rigorous statistical methods. And the answer: a compact and resounding “no”. Perhaps not a surprise – but now backed by testing.

Where AI can and does have a positive impact on our industry was confirmed to be in terms of automation. Incorporating aspects of AI can speed up some analyses and can make some qualitative tasks previously thought to be daunting or impossible easily within reach. TRC has incorporated AI in its text analytics approach for some years, and our newer endeavors to access a wealth of information through a simple query are now being realized. Both approaches contribute to providing our clients with better service and insights.

#2: Data Quality and UX

Fraud is a major issue in our industry, and certainly not ignored at this conference. A couple of talks attempted to put a figure onto its presence and suggested ways to combat it. Much of which TRC is already doing, and some additional actions for us to try out. But also of concern is poor data quality from well-meaning humans. And the bulk of this boils down to researchers treating our respondents, welp, like humans. UX is typically a theory applied to product research and design. But why shouldn’t it be applied to our surveys, which is also a product? It was shown that both mobile and computer-based instruments can be made to provide a simpler and better experience through thoughtful and careful design, which has resulted in an increased quality of responses. Again, TRC regularly takes care to consider the respondent in our survey builds, but more can and will be done.

#3: Important but Invisible Improvements to Current Methodologies

I was thinking about this related to our work at TRC. Our Analytics department spends a great deal of time researching, testing, and implementing improvements to the methodologies we utilize on a normal basis. These can be tweaks to a conjoint design, an adjustment to input parameters, a different rescaling to improve interpretation, or using a slightly different version of a model within the same class. But what is fascinating to me is that these enhancements largely go unnoticed by clients or even most of our internal employees. Our clients trust that we are doing these things, applying the appropriate and current technologies to their studies. It’s a trust not taken for granted, and in light of this, several talks presented a list of aforementioned tweaks that our Analytics team at TRC will be working to implement.

Lastly, I’d like to end with two quick food-for-thought ingredients necessary for growth. First, I appreciate the conference committee’s acceptance and inclusion of a couple “failed” talks. The presenters had a theory, tested it, and were proven incorrect. But there is never advancement without failure and these moments encourage discussion and further investigation. Second, a talk, a tribute, and numerous conversations harkened back on the history of methodological and technological advancements and the people that made these possible. It’s vital to understand and appreciate the past in order to improve in the present and future. TRC is excited to play our small part in contributing to the growth of the market research industry.