Article Review: Trust in Artificial Intelligence

July 2, 2024

artificial intelligence

This post discusses a meta-analysis of studies related to trust in AI systems published by Alexandra D. Kaplan, Theresa T. Kessler, J. Christopher Brill, and P. A. Hancock in Human Factors: The Journal of the Human Factors and Ergonomics Society. As I review it, I consider its implications for how AI might continue to be taken up in work contexts.

One of the things you notice in conversations about AI is how extreme the question of trust can be. Some folks immediately discredit using AI, while others accept it uncritically. Binary thinking is easy when any new technology pops up, but given AI’s radical potential to transform activity it feels understandable, if somewhat misguided.

Personally, my perspective on AI tools is neither wholly positive nor negative. I tend to start from the assumption that they are “are here,” that it’s better to understand them, their affordances, and their limitations than to ignore them. Kaplan et al.’s meta-analysis is useful in that effort.

Article Methodology and Overview

Kaplan et al. analyzed 65 articles in their meta-analysis. Specifically, they included articles that addressed:

Studies that met the following criteria were included:

Across the 65 articles, Kaplan et al. noted four common “delivery systems” for AI:

While the authors noted several limitations in their meta-analysis, their findings pose interesting considerations and questions about trust in AI systems.

Trust in AI

So, what factors affect an individual’s trust in AI? Kaplan et al. highlighted several fascinating responses to that question. It’s useful to point out, though, that the authors didn’t engage the question of whether we should trust AI systems. This is understandable for many reasons, including the nature of their study and the difficulty of assessing AI in general. Personally, I think it’s going to be increasingly less useful to talk about AI as “a thing.” To ask whether we should trust AI, in my view, will soon be like asking whether we should trust websites. “Well… It depends on the site and how you are using it, obviously.”

That said, I am going to take the authors’ lead and offer less of my own thoughts on the trustworthiness of specific technologies. It seems more interesting to think through and play out what their findings tell us about how and why individuals trust/distrust AI systems.

Across the surveyed studies, the authors noted several human factors that were predictors of trust in AI systems. Personality, culture, and identity factors were found relevant. A few examples:

  1. “Lonely participants” in studies were found less likely to trust AI while “innovative individuals” were more likely to pursue its use.
  2. A variety of cultural factors and identities were found to affect trust. For example, Germans were more trusting, whereas Japanese individuals tended toward skepticism.
  3. Male participants were generally more willing to trust AI than female participants (343).

While specific findings may be individually more or less surprising, it seems fairly evident that human factors would play into trust in AI.

One of their findings, though, was more compelling to me because it was something I hadn’t considered. Kaplan et al. noted that an individual user’s technical competency and understanding of an AI system are seen to be positive predictors of trust. Additionally, a user’s expertise in their tasks—what they are trying to do with AI—also factored positively (343). What this suggests, to me, is that a large part of achieving trust in AI systems is increasing knowledge of how they function and fostering environments where goals can be met through their use.

Many technical folks know the anxious feeling of trying to set up a friend or family member with a “cool new tech” and have it not work immediately. (If it’s not obvious, I’m talking about myself here.) The anxiety comes from knowing that while the fix may take seconds, trust in the system is diminished and might not recover. In terms of AI, it seems like the same phenomenon is at play. If one’s goal is to increase trust in an AI system (which, again, I’m not saying should be true in every situation), then creating small-scale positive outcomes seems more important than riskier attempts at paradigm shifts.

Across the 65 articles, there were also many findings about how AI systems, as technology, factored into user trust. The reliability and performance of an AI system were highly predictive of trust (344). More than that, though, the visibility of possible failure or negative outcomes is correlated to trust, as well. Because of the possibility for AI systems to “exhibit undesirable behavior,” humans need “properly calibrated trust in AI systems [that] highlight the potential for negative outcomes” (347).

This visibility may be increasingly difficult as AI systems continue to develop, though. The authors warned that AI may continue to develop as black boxes to users. While this has clear issues related to human agency, a corollary concern is that it will decrease trust in AI systems.

While current iterations of AI do not have an internal drive to misbehave, lack of transparency in some systems may seem to indicate deception to some users. Self-repairing code and learning systems that generate new code based upon experience do not require a user’s input. Systems will, therefore, become increasingly ‘black boxes,’ leaving the human beyond the loop on decisions and may subsequently then be (mis)interpreted as devious or deceptive. (347)

Trust doesn’t just depend on success, but an understanding of possible points of failure/limitations. But this understanding may be made difficult by technological development.

Trust in Context

Finally, the authors discussed contextual factors related to trust in AI. For example, the “personality” of an AI system was relevant to the amount of trust it provoked. People preferred AI systems that encouraged collaboration but, maybe obviously, distrusted ones where they perceived elements of deception (344). They also preferred voice-based communication over text (344-45).

The reputation of the AI system was also found to drive trust (344). This finding calls up the question of marketing of AI systems and how that might transform their uptake. What feels so significant about Apple’s partnership with OpenAI is that it gives the latter company access to Apple’s fairly stellar reputation. For better or worse, the marketing of “Apple Intelligence” (see the video below) feels much more… reassuring than OpenAI. I sense that this will lead to more widespread adoption of and trust in AI tools.

Conclusion

I’ve highlighted what I see as the main implications of Kaplan et al.’s findings above, but here is a quick review.

Again, I’m purposely avoiding whether we should trust a given AI system because that question is simply too large to answer in general. But, to the degree that AI tools will be accepted and reshape labor, it’s worth being aware of the reasons that individuals trust them, or not. I definitely recommend Kaplan et al.’s meta-analysis for more detail on the topic.

References

Kaplan, Alexadra D. et al. “Trust in Artificial Intelligence: Meta-Analytic Findings.” Human Factors, vol. 65, no. 2, pp. 337-359.