“Too Confident To Correct?” How Generative Ai Reliability Cues Shape User Overconfidence

This research investigates how the reliability of Artificial Intelligence (AI) affects human confidence in the decision-making process. The study used an experimental design, recruiting 192 participants (N=192) who were assigned to three controlled conditions (No AI, 0%; AI with 75% reliability; and AI with 95% reliability). The research process comprises 10 General-knowledge true-false questions, which were provided to the research participants. Participants were provided with AI-generated suggestions for all true and false statements, along with their assigned reliability level. This research study aimed to evaluate the impact of genAI reliability and the presence of AI support on participants’ confidence in their decisions, regardless of decision accuracy. The results of ANOVA indicate that AI assistance significantly affects research participants' overconfidence. Participants in all controlled conditions, including 75% and 95% AI reliability, observed that AI was more accurate and reliable. The findings of this research study also undermine the psychological impact of AI suggestions on human judgment, which is critical to designing an AI system.