As more healthcare systems turn to AI scribe technology that listen in on clinic visits and draft doctors’ notes, what is happening to the original audio recording and subsequent transcript of the patient-clinician conversation?

An editorial in The New England Journal of Medicine last month suggests that many US healthcare organizations are deleting those transcripts after a doctor reviews and finalizes the clinic note, at least in part, authors said, because of concerns over malpractice liability.

The destruction of millions of clinical visit transcripts is “unacceptable,” authors Katherine Goodman, JD, PhD, assistant professor, and Daniel Morgan, MD, of the Department of Epidemiology and Public Health at University of Maryland School of Medicine, Baltimore — wrote in the piece.

Deleting these transcripts will make it more difficult to evaluate the validity of current and emerging AI technology and hinder research in general, as the transcripts could provide a basis for key studies on patient language and how patients report and phrase symptoms, Goodman told Medscape Medical News.

“It’s just this idea that healthcare systems may be caught a little bit flat footed if they don’t at least engage with this idea of should we be preserving transcripts,” said Goodman, who used to practice law.

‘Call to Action’

AI scribe technology — such as Abridge and Nabla — are being adopted across US healthcare institutions at an “unbelievable pace,” Goodman said, but what she and Morgan concluded was that these transcripts are “extremely difficult” to obtain and most of them are ultimately destroyed.

Barriers to convincing healthcare organizations to keep the transcripts are much bigger than any one researcher or clinician can solve on their own, Goodman said.

Therefore, the primary motivation for writing the editorial was “…hopefully a call to action of we’re going to need different stakeholders to come together to highlight the value of these transcripts and then design strategies to ultimately preserve them,” Goodman said.

Why Save the Transcripts?

One of the most immediate reasons to save the transcripts, according to Goodman, is to be able to test the validity of the AI technology. It is “critical to understand, for example, something as simple as what is the frequency with which these tools hallucinate information or insert incorrect information into a clinical summary…?” Goodman said.

There are no studies just looking at how accurate they are, Morgan said.

There are some additional factors to be aware of. For example, there are probably “different motivations for AI scribes,” Morgan said, “…and they’ve been optimized to improve billing.”

In fact, some data show that we are billing for more now that we have these scribes, he said.

Pursuing new research avenues is another reason to save the transcripts, Goodman said. Take research on early symptoms potentially preceding a diagnosis of ovarian or pancreatic cancer, for example. Patients with these conditions often have no or few symptoms — or very nonspecific symptoms — until late-stage diagnosis, Goodman said.

“But we’ve never really been able to pressure test that,” she said. “And it may be that there are subtle symptoms expressed years before diagnosis that we just have not captured. But if you had the transcripts from your visits throughout that 5-year interval preceding diagnosis, you…could do that type of study.”

“It’s not hyperbole to say that transcripts and being able to memorialize the clinical conversations and the patients’ actual words, it opens up an entirely new avenue of research especially around…understanding how can we reduce diagnostic errors, diagnostic delays, misdiagnoses,” Goodman said.

Transcripts in Malpractice Cases

Goodman and Morgan based their conclusion — that most healthcare organizations are not keeping the transcripts — on conversations with colleagues at other institutions, with product vendors, and through discussions with national research and legal networks. They were “all somewhat backchannel or informal sources,” Goodman said.

“Institutions are not sort of formally sharing their policies, but through informal networks…our understanding is most — if not all — are not retaining these,” she said.

While they cannot prove it, Goodman’s and Morgan’s understanding is that for healthcare systems, the transcripts are primarily viewed as having greater risks around malpractice than any research benefits, they might offer.

There is some value in retaining the transcripts for sociomedical and physician/patient-interaction studies, but “my gut feeling is that society is not missing out on a boon of critical research findings in the absence of these transcripts.” Walter Sujansky, MD, PhD, president of health IT consulting firm Sujansky & Associates based in Woodside, California, said in an email.

Sujansky said retaining the transcripts would be valuable for certain research purposes and for improving the accuracy of ambient-scribing technologies. However, at the same time, he understands healthcare providers’ concerns since these transcripts would “almost certainly” be subject to discovery in medical malpractice cases.

He agrees with Goodman and Morgan that to win the buy-in of healthcare organizations, it’s important to provide “safe harbor” protections to the ambient-scribe transcripts that would exclude them from discovery in litigation.

Laura Zwaan, PhD, associate professor at the Erasmus Medical Center, Erasmus University Rotterdam in Rotterdam, Netherlands, has a different take. It’s debatable whether healthcare organizations that kept the transcripts would lead to more malpractice cases and that notion should be further explored, she said.

“Patients in hindsight, think often differently about…the consultation that they had. So maybe if they listen back and they see that the doctor listened and they hear what was actually discussed, they might think more positively about it…,” Zwaan said.

And there is another point Sujansky wonders about:

“Would healthcare providers and/or patients, themselves, want there to be a verbatim record of every conversation regardless of whether that record was subject to discovery in litigation?” Sujansky said.

Salaried physicians are feeling “more and more managed in their clinical duties, and their employers’ use of such transcripts might make them feel even more like customer-service representatives at a call center rather than highly trained professionals,” Sujansky said. Likewise, patients may feel reluctant to tell physicians certain things if they knew a “mandatory verbatim record of the conversation” was potentially available to insurance companies, employers, and family members, he said.

Who’s Keeping the Transcripts?

Hospitals may not keep these transcripts, but that doesn’t mean companies aren’t preserving them in some way, Goodman said. However, “we don’t know how all companies are handling it because no one’s really willing to speak on the record.”

Companies “are receiving valuable dialogue to train the next generation of conversational clinical AI tools,” Goodman and Morgan wrote in the piece. Their understanding is that after a certain period — such as 30 days — companies may be using automated algorithms to strip out the health and patient identifiers from the data, Goodman said.

Three years ago, medical researchers were more likely to develop AI tools “and they would report exactly what the tool did and how it works in public and then companies would develop that further into a product,” Morgan said. And you understood how well it worked. But now things are different. Companies are selectively sharing positive vs negative data, he said.

“I think it means that…there are likely errors in some of these AI tools, that there are hallucinations or omissions, and that some tools probably work better than others, but we have no way of knowing…,” Morgan said.

Additionally, because AI scribes don’t qualify as medical devices, their widespread adoption is occurring without routine FDA oversight, Morgan and Goodman wrote in the piece.

Next Steps

Goodman said she would like to see an open dialogue about what it would take for healthcare systems to conclude that the benefits of preserving the transcripts outweigh the risks.

“How are we going to develop the standards and the infrastructure to do that in a way that patients are comfortable with?” Goodman said.

One first step could be to strip patient identifiers from the transcripts. Doing so may limit research value, but it would also likely mitigate some of the malpractice risks, she said.

The pathway forward may also involve the creation of new laws — “potentially a malpractice shield that says if you preserve these transcripts for research use, that’s in the public good for X number of years, they would not be disclosable in a malpractice lawsuit,” Goodman said.

And it’s important to figure these details out sooner rather than later, she added.

Currently, AI ambient scribe tools are just producing visit summaries, Goodman said. But these companies have said on record that they are working on follow-up tools that might suggest testing orders or draft prior authorizations for doctors. Eventually, they may suggest diagnoses and further clinical workups, Goodman said.

“Healthcare systems need to figure out what they’re doing with these transcripts because by a year or two from now, they’re going to be the foundation for sort of so many other…diagnostic support tools that will raise much higher risks if we don’t know how well those tools work,” she said.

For Morgan, his hope is even more immediate.

“My greatest hope I guess is that people see the importance of these and that there are a few institutions that may be saving these and that they announce that publicly,” he said.