Beyond a Reasonable Doubt? Evaluating User Trust and Reliance on AI-Generated Legal Advice

Abstract

As large language model (LLM) chatbots become increasingly accessible, concerns have emerged about users relying on them for professional advice in high-stakes domains like law, given their tendency to produce inaccurate, misleading, and biased responses. While a small body of existing research suggests that many people are willing to consult chatbots for legal advice, little is known about what the nature of these interactions might be, users' willingness to trust and act on the information they receive, and how various factors influence perceptions. We investigate these questions through a large-scale survey experiment involving over 900 participants. Ultimately, we find that 21% of participants had previously used an AI chatbot for legal advice, and that the vast majority considered the advice credible and used it to take action. After assessing participants' prior use of AI chatbots for legal advice, we randomly assigned them to one of five treatment groups. Each group, with the exception of the control, was exposed to either a warning message (weak, medium, or strong) or a marketing claim modeled after real statements made by OpenAI about GPT-4's performance on the Bar Exam. Participants were then presented with a legal scenario, asked about their initial intended course of action, prompted to interact with an embedded chatbot for advice, and assessed on their perceptions of this advice. Contrary to our expectations, statistical analysis did not reveal a significant relationship between the marketing claim we test and user trust. However, qualitative evidence suggests that it may have influenced trust at the individual level. Similarly, warning messages - styled after those on leading chatbot interfaces - did not appear to significantly reduce trust or deter users from relying on chatbots for sensitive legal issues. Stronger warnings showed modest, though inconclusive effects. These findings carry important implications for the degree of legal liability developers may face. Design and marketing choices that overstate the capabilities of chatbots or otherwise fail to inform users about their limitations might expose developers to liability under doctrines such as negligent misrepresentation and consumer protection law. Based on these considerations, we develop a set of policy recommendations that would simultaneously help limit the degree of liability faced by developers and protect users from the risks of overreliance on AI-generated legal advice.