Leo Sher, M.D.

The article, “Human in the loop: Balancing artificial intelligence, clinical judgment, and legal responsibility” was recently published in the International Journal of Law and Psychiatry (1).

This article examines the growing role of artificial intelligence (AI) in psychiatry, mental health care, forensic psychiatry, and legal practice, while emphasizing the limitations and responsibilities associated with its use. The author acknowledges that AI has considerable potential to improve risk assessment, diagnostic support, case triage, and legal analytics through its ability to process large datasets and identify complex patterns. AI tools are already being explored for violence risk prediction, mental health monitoring, judicial decision support, and other applications. Despite these potential benefits, the paper argues that AI fundamentally generates predictions rather than reasons, distinguishing it from human professional judgment.

The author contends that medicine and law are grounded in justification, accountability, ethical reasoning, and interpretation, all of which remain uniquely human responsibilities. They identify four major tensions that arise when AI is integrated into clinical and legal practice: epistemic, ethical, relational, and legal tensions.

The epistemic tension concerns the “black box” nature of many AI systems. While AI may produce accurate predictions, its internal reasoning is often opaque, making it difficult for clinicians and lawyers to explain or justify decisions based on its outputs. This lack of transparency threatens trust, accountability, and professional legitimacy.

The ethical tension arises because AI systems are trained on historical data that may contain biases. Consequently, AI can perpetuate or amplify existing social, racial, or economic inequalities. In mental health and legal settings, biased predictions may influence high-stakes decisions such as involuntary treatment, sentencing, or parole, potentially causing significant harm.

The relational tension reflects the fact that psychiatry and law are fundamentally human and interpersonal professions. Effective clinical care and legal advocacy depend on empathy, trust, narrative understanding, and moral dialogue. Although AI can simulate conversation and provide information, it cannot genuinely understand or experience human emotions, relationships, or lived experiences.

The legal tension emphasizes that responsibility cannot be delegated to algorithms. Clinicians, lawyers, judges, and institutions remain accountable for decisions made with AI assistance. Courts and regulatory bodies increasingly require meaningful human oversight, transparency, and governance of AI-supported decisions.

To address these challenges, the author proposes the HUMAN framework. The first principle, Human Judgment is Primary, asserts that AI should support but never replace professional reasoning. Human decision-makers must remain the final arbiters of clinical and legal actions.

The second principle, Understand the Model, encourages professionals to understand the data sources, assumptions, limitations, and potential biases of AI systems. Users must critically evaluate AI outputs rather than accept them uncritically.

The third principle, Monitor Performance, stresses the need for continuous evaluation of AI systems. Models may become less accurate over time due to changing populations or circumstances, making ongoing monitoring essential to detect errors, bias, and unintended consequences.

The fourth principle, Accountability Stays Human, reinforces that legal and ethical responsibility remains with people, not machines. Institutions must establish governance structures, audit mechanisms, and documentation practices to ensure responsible AI use.

The fifth principle, Narrative Matters, emphasizes that individuals’ stories, values, experiences, and preferences must remain central to decision-making. The author argues that AI cannot truly understand human narratives, whereas clinicians and lawyers are uniquely equipped to interpret and respond to them.

The paper concludes that AI should be viewed as a tool that can enhance professional practice but cannot replace the human qualities that define medicine and law. The author argues that compassion, dignity, fairness, empathy, accountability, and moral judgment remain indispensable. They advocate for cautious and ethical integration of AI, supported by research, transparency, regulation, and interdisciplinary collaboration. Ultimately, the article maintains that the future impact of AI will depend not on the technology itself but on how humans choose to use it. The HUMAN framework is presented as a practical guide for ensuring that AI strengthens rather than undermines the human foundations of care and justice.

Reference:

1. Kelly BD. Human in the loop: Balancing artificial intelligence, clinical judgment, and legal responsibility. Int J Law Psychiatry. 2026 Jul-Aug;107:102226. doi: 10.1016/j.ijlp.2026.102226. Epub 2026 Apr 2.