The future of customer research in a distrustful AI world

This article is a summary of a speech given by our Executive Director, James Garriock, to the Australian water industry. It applies to all institutions and natural monopolies.

As artificial intelligence (AI) continues its rapid integration into various aspects of our lives, it prompts the question about implications for the future of customer research. This article outlines the impact of AI on customer research, whilst delving into potential strategies to effectively navigate the complexities of conducting customer research in a climate scepticism fuelled by AI.

AI’s role in shaping customer research

1. AI-enhanced interviews for deeper insights

AI is going to revolutionise customer research. It can “discuss” a topic with a respondent in an interview and modify the prompts in real time to delve deeper into the drivers of an opinion. Admittedly, it is difficult to “train” but Insync is already working on it. Further, AI can crunch qualitative data rapidly to answer questions like “Looking at the full interview scripts of people who rated lower than 7 out of 10 for Satisfaction, what differences are apparent between female and male responses?”

2. Analysing sentiment through voice-to-text

AI’s capability to convert spoken words to text is a game changer. At Insync we are trialling online focus groups where people discuss things in real time in different languages. Additionally, in one-on-one interviews, AI can deduce the sentiment of a person’s response by listening to the tone of voice being used. There is also a plugin where the AI can isolate each person’s voice in a room, not just transcribing what they say, but attributing it to the speaker. During analysis, this means that “Person A”’s comments can be considered in sum.

3. Data cleansing and exploratory analysis

One of the biggest productivity applications of AI in market research is in data cleansing and exploratory analysis. These are the tasks that you, as customers, do not see, but they will change the employment landscape in the industry and drive prices down.

Challenges in modern customer research

Taken at face value, the productivity and insights leap that AI promises should mean that customers can look forward to getting more for less in future. The reality is that human behaviour might result in you getting fewer insights than you get now.

1. The dilemma of technological ease

Frankly, customer research has been getting more difficult ever since it started getting easier. When fieldwork was difficult, organisations only did customer research when it really mattered, so people didn’t tend to get pestered for their opinions very often. With the advent of the smartphone and the internet, fieldwork started getting easier, and that’s when it started getting harder. Samples became unreliable; for example, we all find it hard to get enough young people, small business owners, and Aboriginal and Torres Strait Islanders in our studies.

Methodological concerns started to cast a long shadow of doubt over the population differences when we compare studies: I find myself spending far too much time thinking about selection bias, question type, response scales, and technologies than I’d like to. None of the time spent criticising our work adds any value to yours.

2. Diminishing response rates

Response rates plummeted. For example, in our work for the Water Alliance, a group of seven leading Victorian water corporations that commission their research together, and share their results, the number of phone calls required per response has gone from 15 to 24 in the last two years.

Not only that, but I’m increasingly being asked “What kind of person answers the phone AND then does a survey?” By definition, if only a tiny minority of people give us feedback, then we can safely say that the responses are NOT representative.

3. Obstacles in traditional approaches

I believe that outbound surveys will be dead very soon and that the speed of their demise will surprise everyone.

An AI can keep a real person on the phone for half an hour even now. That’s when AI puts voice-to-text in the chatbox and then converts text to voice on the other side. People can’t quite tell whether it is a real person, and that experiment was before GPT. GPT 3.5 would be better than that and 4.0 would be another order of magnitude better.

So, at some time in the next couple of years, it’s inevitable that phone scammers as well as well-meaning researchers will think, “I can get rid of the 100 people in my call centre and replace them with an AI”. Instead of making 10,000 calls an hour, it will be ten million calls an hour. Within weeks, the system will be overloaded with calls and the behavioural response will be that nobody in Australia will pick up their phone. What would happen to your metrics? All those wonderful lines on the charts, that you have been showing to your board for years will come to a full stop. Is that moment years away? Or months? Or weeks?

Even before the AI tsunami, the unrelenting disclosures about scams and data breaches have made the public wary of participating in anything, at any time.

You might think that we can move to another method, for example, QR codes have been in the ascendency since the pandemic. However, scammers make fake menu websites and put their own QR code stickers over legitimate QR codes on restaurant tables. You scan the menu, pay your money (to the scammer) and the food never arrives.

I simply don’t think we can feel comfortable that our customers will continue to scan QR codes to provide feedback. The trend of QR code usage may follow that of inbound call answering.

Our next fallback option is likely to be EDM surveys. These are already becoming problematic. “DO NOT CLICK ON A LINK” is the mantra now. This is not likely to go away. Through our work with regulated utilities doing price submissions, we have watched corporations send out 50 or 100 thousand invites and get fewer and fewer responses.

Crucial differentiation for meaningful feedback

It’s relevant for utilities to ponder the question “What’s worse than being asked for your opinion by your monopoly provider? Answer: Not being asked for your opinion by your monopoly provider.”

The requests from an institution or natural monopoly provider might look the same as market research by Pepsi, your recent purchase of a pair of socks or how well you slept during a recent hotel stay, but it is qualitatively different. You can drink Coke, (or water!), go without socks or stay somewhere different; but obviously, you’re a captive consumer of your Waterco. What that means is that the time a customer spends providing their opinion is a social good, like paying taxes, stopping at a red light or doing jury duty. The other types of market research are an altruistic act, a gift of time to help one private company out-compete another.

If utilities, collectively, want good quality feedback from the people they serve, then they are going to have to invest some time and effort in making the distinction between themselves and the private companies.

In summary, AI is going to make analysis easier, but it is going to make fieldwork harder, and without responses, it doesn’t matter how clever your analysis is. Furthermore, whilst utility requests for feedback look the same as everyone else’s, natural monopolies are quite different, and if we don’t make the distinction then we are in for a very tough time.

Potential solutions

In my view, there are (at least) three approaches that could address this looming disaster. We simply do not know when step changes in AI capability are going to cause our current response numbers to crash. If we are not proactive and if we don’t work as a team, then down the track, things are going to get messy.

So here are my approaches, and I’d encourage industry debate and additions.

1. Triangulation

Triangulation is where we use multiple techniques to increase our confidence that the results we are seeing are indeed representative. For example, in price submission work we use narrative techniques to uncover values, then surveys and focus groups to explore them, and then valuation techniques to prioritise them, and deliberative techniques to provide assurance. The techniques each have their flaws, but together they provide greater confidence. Of course, sometimes they contradict one another, which means even more work, or better still, the wisdom of the crowd in an empowered deliberation.

2. Revealed preferences

In research, we tend to look at stated preferences and revealed preferences. For example, a stated preference would be asking people in a survey how quickly they want their Waterco to answer the phone; whereas a revealed preference is looking at the dropout statistics and post-call satisfaction data.

Recently, instead of asking people how much they would pay for better-tasting water, we asked them about their private expenditures on filters and rainwater tanks to value palatability improvements (revealed preference).

Instead of asking them whether they would pay $2 per bill for extra carbon abatement, we let them tick a box on their bill to reveal their preference for carbon abatement.

3. Leveraging monopoly status

Most of us are aware of triangulation and revealed preference. What we’ve devoted less time to is considering the third approach, leveraging our unique status as a monopoly provider to establish a social contract with our customers. This isn’t something that your research partner can do and is best coordinated at the national level. Remember, all we are aiming for is getting good quality, representative feedback.

There are four ways Insync has identified to help leverage the special status that DNSPs, water corporations and government agencies have:

1. To get people to respond to calls, whether IVR, AI or real humans, send them a text saying that you’re about to call. This technique would be even more effective if it came from an authenticated “from” phone number so that when you, or your regulator calls, the consumer can trust the call. I would suggest that the industry talk to the telcos about whether your “from” number can be authenticated by the telco, and if such a scheme does not yet exist, then lobbying at the federal government level is immediately required. I can imagine that all governments would like a gold standard ability to contact the people they serve.
2. On the email side, every utility needs to establish its credibility by email with regular communications. Behavioural acclimatisation is necessary to build trust and normalise the practice of clicking on a link in the email from your Waterco or DNSP. As soon as you are trusted, however, then your identity is a target for scammers. You’ll need to insert measures to prove your identity which scammers can’t follow.
3. Away from phone and email, utilities can use digital meter rollouts to encourage the adoption of an app – and get your feedback through that app. Water is probably less engaging in this regard than energy
4. Finally, let’s think about going analogue. Back to the future. Send customers a letter in the mail (or a bill insert). Yes, paper is not dead, and it doesn’t have malware installed either! The letter/postcard can have a QR code to an online version of the survey and a phone number for an interactive, AI-led interview, a website address to type in, a way to download your app, and the actual survey questions that can be posted back.

In conclusion, we are driving towards a precipice in the dark. I don’t know exactly when we’ll reach it, but I’m certain that it lies in wait. In this article, I’ve provided ideas for calls, emails, apps, and paper, but none of these technologies are a silver bullet. What needs to happen is to re-cast the act of providing feedback as a social obligation, not an act of altruism. There is no narrative around this at any level of government at present, and I believe it is a great example of where the utilities can take the lead on behalf of all institutions in Australia. Organisations like the Water Services Association of Australia and the Energy Charter are clear candidates for that leadership. I like to imagine a time when providing feedback to institutions is normalised, expected, and valued. A time where essential service providers are valued not ignored, because that’s going to lead to better outcomes for all consumers. That change isn’t going to happen by accident, but the time is now.

If you wish to delve into this topic more deeply, don’t hesitate to get in touch.

About the author