A physician signs a progress note generated from an ambient recording. Another accepts a predictive risk score embedded in the electronic health record. A third submits an artificial intelligence (AI)-drafted appeal of an insurance denial under her own name. In each case, the organization calls the technology “assistive.” But who exercised judgment? That question will define the next stage of healthcare AI, and we are not asking it clearly enough.
The loudest fears about AI in medicine imagine a dramatic handoff: a hospital announcing that algorithms now diagnose and physicians are obsolete. That is not the danger I worry about most. The quieter and more likely one is that clinicians will continue to carry legal, ethical, and professional responsibility for decisions while gradually losing the time, information, authority, and discretion required to make them.
We should stop treating “AI use” as a single category. Ambient documentation tools shape the clinical record. Predictive systems direct attention toward some risks and away from others. Generative tools draft prior-authorization appeals, patient instructions, and discharge summaries. Message-triage systems decide what a clinician sees first. Each of these changes the encounter in different ways, and each raises distinct questions about consent, review, and accountability. Lumping them together obscures the hard questions.
The common thread is a widening gap between where responsibility sits and where control lives. Medicine has long tied accountability to identifiable professionals: physicians sign notes, enter orders, and answer for the consequences. AI complicates that arrangement in a specific and troubling way. A clinician may sign a note that was largely machine-generated, technically accurate yet stripped of the uncertainty, hesitation, or social context that made the encounter clinically important. A physician may accept a predictive recommendation because it looks objective, because the schedule leaves no time for independent review, or simply because overriding it requires extra documentation while accepting it requires a single click. A denial appeal may go out under a clinician’s name containing arguments she never had time to examine.
In every case, formal accountability stays human while practical control becomes technological and organizational. That is an unstable and ethically questionable arrangement. A person cannot be meaningfully responsible for a decision unless she has the authority, information, and opportunity to make it.
This is where automation bias becomes dangerous. The worry is not that clinicians will blindly obey algorithms. It is that organizational conditions — time pressure, staffing shortages, fragmented records — will make independent review feel inefficient and eventually optional. When AI-generated documentation is usually correct, reading every sentence starts to seem like wasted effort. When a recommendation is built into the record rather than offered as one opinion among many, it acquires an authority it hasn’t earned. Over time, clinicians risk becoming reviewers of machine output rather than originators of professional judgment. A visibly absurd error is easy to catch. A polished, coherent, clinically plausible error can enter the record, influence later decisions, and gain clout through repetition.
None of this is an argument against the technology. Ambient tech can reduce the documentation that drives burnout and let physicians look at patients instead of screens. Predictive tools can identify deterioration, adverse drug events, or patients who need follow-up. AI-assisted appeals can help clinicians push back on an insurance system built to exhaust them — the same system that often uses AI to generate the denials in the first place. These benefits are real. But efficiency alone cannot tell us whether a tool is governed well, and prediction is not the same as judgment. A model is built from past data, selected variables, and a defined outcome; it can tell you what has happened before, but it doesn’t have the clinical acumen to reliably say what will happen next — much less what should happen for the patient in front of you. A risk score should start a conversation, not end one.
Two safeguards deserve particular emphasis.
First, consent must be specific. Patients cannot meaningfully consent to “AI” in the abstract. They should know whether a visit is being recorded, whether the recording is retained, what the system produces, who reviews it, whether the output will influence diagnosis, treatment, discharge, or insurance authorization, and whether declining will affect their care. Broad disclosure that “AI may be used” protects the institution more than it informs the patient.
Second, clinicians need to pay attention to what a tool cannot see. Tone, contradiction, family dynamics, and the quiet sense that something is not right rarely survive a clean automated summary, yet they are often what matters most.
What healthcare organizations need is a plain standard: assistive, not autonomous. In practice, that means every consequential AI-supported process has a named accountable human and a defined scope — whether the system drafts, recommends, predicts, or decides. It means human review that involves more than clicking “approve,” with enough time and information to change or reject the output. It means visible disclosure to patients and clinicians, a practical way to correct errors before they spread through the record, and protected override authority so a clinician can depart from a recommendation without retaliation or a mountain of extra paperwork. And it means governance that evaluates more than technical accuracy — whether consent is meaningful, whether the tool’s effects land equitably across patients, how it changes daily workflow, and what it does to the clinical relationship. It also requires honest monitoring of whether clinicians are actually reviewing outputs or just approving them under pressure.
The public debate on healthcare AI tends to split between enthusiasm and alarm. The more useful position sits in between. AI can help clinicians see more, remember more, and reason more effectively. It should not be used to simply manufacture the appearance of human judgment. A clinician’s signature cannot become a ceremonial act placed beneath a machine-produced decision. If physicians and other clinicians are going to remain accountable, they must also remain meaningfully in control. AI should be assistive, not autonomous — and clinical judgment must remain more than the final click.
