Summary
“Every physician has thought this idea. Every physician has thought, I should not have to write my own notes.” Dr. Raj Bhardwaj sits down with Dr. Jared Pelo, the co-creator of DAX Copilot, to delve into the origins and future of the AI-powered clinical documentation solution. Discover how DAX Copilot aims to revolutionize medical workflows, the challenges it faces, and the potential it holds for transforming patient care. Join us to hear insights from the creator himself on the evolving role of AI in healthcare and what lies ahead, as Dr. Pelo candidly addresses questions about AI accuracy, patient privacy, and the future integration of AI in healthcare.
Visit aka.ms/ddx-podcast to view a DAX Copilot demo today.
Dr. Jared Pelo, M.D.
Jared Pelo, MD, is an Emergency Medicine physician currently practicing Direct Primary Care. He is also an entrepreneur and technology innovator with a focus on transforming healthcare through ambient AI and digital solutions. As a pioneer in the development of one of the first Ambient AI systems, Dr. Pelo played a crucial role in building and scaling Microsoft’s Dragon Ambient eXperience (DAX) Copilot, revolutionizing clinical documentation by leveraging AI to streamline workflows and reduce administrative burdens for healthcare providers. He has worked to integrate advanced AI into healthcare, promoting solutions that enhance patient care and provider efficiency. Dr. Pelo’s experience combines clinical expertise with a profound understanding of how technology can reshape healthcare delivery.
Transcript
DDx SEASON 11, EPISODE 6
The Human Behind The AI In The Room
RAJ: This is DDX, a podcast from Figure 1 about how doctors think. I’m Dr. Raj Bhardwaj.
Over the last five episodes, we’ve looked into DAX Copilot, an AI-powered clinical documentation and workflow assistant—what it promises, how it’s performing, and the concerns it raises.
We’ve spoken with physicians who use this tool every day, uncovering how it truly works in practice, and where challenges still exist.
But these systems are complex, and questions remain. Today, we’re doing something different.
We’re going straight to the source.
Dr. Jared Pelo, a physician and health tech entrepreneur, co-created the technology that became DAX Copilot, now owned by Microsoft.
He’s here to answer the questions only a creator can—about what this system can actually do and what the future holds for AI in medicine.
Here’s our conversation.
RAJ: How did you come up with the idea that became DAX Copilot?
DR. PELO: Every physician has thought this idea. Every physician has thought, I should not have to write my own notes. Computers someday should write my notes for me. So the idea wasn’t the hard part. It was, how do you actually take that idea, build a real business, get doctors to buy it, and thenwait around long enough for the artificial intelligence to catch up, to make it possible.
And now we’re talking about an AI copilot that will only get smarter over time and only get more capable. And that’s what I think is so exciting is realizing we’re really in the beginning innings of this and these tools will only become better for us.
RAJ: Do you find that the patient that you’re talking to responds differently if there’s a human medicalscribe there versus if they know that the conversation is being heard by this AI scribe?
DR. PELO: Remarkably, I haven’t seen much of a difference. We’ve been recording people for over adecade now and there’s always been concerns from doctors on many fronts, like will patients want to be recorded? And overwhelmingly if the patient feels like they’re going to receive better care, they don’t mind being recorded, all they want is really good medical care.
And then the other thing you’ll hear doctors concerned about is: if I record this, then is it going to show up in a court of law? And in a decade of recording people in 15 million plus recordings, we’venever had any recording show up in a court of law.
Now, of course, we keep the data for such a little amount of time. It would be nearly impossible for it to show up.
But yeah, I haven’t run into any times when a patient has acted very differently for either a human medical scribe or an AI scribe.
RAJ: So can you explain to us a little bit about what happens in the black box? A doctor goes in, they introduce themselves and introduce the fact that they’re going to be using this AI-driven scribe. The conversation is recorded, and then what happens, between that and the note showing up in the chart?
DR. PELO: The black box is, we have 15 million of these doctor patient conversations growing everyday and we have a model specifically trained on how to write medical notes for U.S. specialties and now starting to be international specialties around the world.
About two years ago, our quality got high enough, we started feeling very confident. We were testingdirect to physician. And chat GPT was released and the world changed. And now there was competition that could start building tools that look similar in what it could do. But when you pull back the covers, you start looking at how medically accurate is it? What are the benchmarks we use in this space? I think that’s something we’re still arguing about.
RAJ: So you brought up chat GPT, and I think that when people think of AI, that’s what they’rethinking these days, interacting with things like that. And one of the things that’s been in the news a lot is these AI hallucinations that happen. And I can imagine people being very worried that somehow,some hallucination,
however small, might creep its way into their medical chart and what a profound impact that could have. Is there some sort of way to make sure that doesn’t happen? Or is that even possible – the waythat the technology is being used?
DR. PELO: Yeah, there are several places where error can be inserted into a medical chart. And Iwould say that’s with a human or without a human, with only an AI, or no AI at all. And so the risk doesn’t go away of error in the medical chart using an AI, just like you have that error with or withoutan AI.
I remember a case in medical school that, we had our morbidity and mortality conference and theresident had documented a normal physical exam and it was somebody with bilateral below the knee amputations. And that was a big deal.
And those kinds of mistakes, whether you say BKA or DKA and you put it in the chart, you dictate it in the chart. Those kinds of mistakes are important to note.
I don’t know that AI as it is today really gives us much more of a risk than we have at baseline. And I dobelieve that as AI becomes more intelligent and more trustworthy, the actual risk of error in the medical chart will only decrease.
RAJ: And so that example that you use of DKA versus BKA, is that something that the AI can actuallycatch now and say one of these doesn’t seem in context, so let’s assume it’s the other one or something like that.
DR. PELO: Yeah, I think we’re exactly at a point in time where you can start to think of these copilots that you can task with looking at different things, like context, look back over the transcript and tell mewhat seems incongruent about my note and my transcript and where I should be looking more deeply. And so I think those are the exact types of problems that AI can be used to solve.
RAJ: And then there comes to the point of where does the buck stop kind of thing… Because, if I chart something, this person who’s had a bilateral leg amputation. And I chart in my note that their physical exam is completely normal. Then the buck stops with me.
What happens if there’s an error that is now made in an AI generated chart, and the recording’s gone because, like you said, you keep it for such a short time.
Has that happened? Is that something that you’ve come across?
DR. PELO: We haven’t had that happen, but the buck does still stop with the physician. In the end, you’re the one signing the note. You’re the one
documenting on the patient. And it’s something that we’ve talked about for years.
Really, you need the physician to pay attention. And how do you have a physician pay attention whenthe physician is given more tasks to do and less time to do it?
And so I think it’s by freeing them up from the tasks that are currently taking too much time. And that’s one of the hopes with ambient scribes and ambient copilots is that we can start freeing the physician up so they can actually pay attention to delivering better care and also making sure their documentation is correct.
RAJ: So on that same line of accuracy, is there also some reproducibility with DAX Copilot? If you fed it the same exact conversation would it come up with the same note? Or is it like when you askChatGPT the same question twice, it’ll actually, there will be differences in its answer.
DR. PELO: Yeah, you should still think of this like a large language model. The interesting thing about large language models is you should compare them somewhat to very intelligent humans, rather than programmatic engineering processes. And so, if you were to ask me the same question twice, I would give you slightly different answers, but the same content. And so we benchmark all of our models and we look at accuracy and it’s really around clinical accuracy.
Are we hitting the clinical accuracy mark, but how things might be phrased, those things might change.
RAJ: And in terms of that accuracy, I can think of cases even just from this week where, I documentedmy note and then, this person got admitted to surgery or internal medicine or whatever, and then theydocumented their note and there are some glaring differences in their note versus my note. And I know mine’s, I know mine’s right, yeah.
In terms of the language that it’s hearing and the output that it’s generating, a lot of my time with a patient is, it’s a lot of subtlety. If I ask, ‘Have you had this chest pain before?’ And they look away andthey’re like, ‘Not really’ and I know that, okay, follow that up. And I will still know that even though DAX will be recording things. If there’s subtle emotional nuances and things like that, is there a way to capture that with a copilot type thing?
DR. PELO: Yeah, so for users, for physicians, we do have to be trained somewhat on how to use thesetools. So if somebody says, ‘It hurts right here’
and they’re pointing at their left shoulder as a physician, I need to say, ‘Oh, your left shoulder is hurting.’
And so if you noticed a nuance, you might need to lean into that. But where we’re at is so exciting in that now we’re building in technology that can screen for Parkinson’s and depression and yourcongestive heart failure and things like that from your voice.
And those will be technologies that we roll out over the next year that will be widely available. And so I think yes, as a clinician, you need to be a little bit more explicit because we want to capture theactual data. We’re not going to write something that isn’t captured. And then what else can we usethat voice biomarker to help us understand?
RAJ: Yeah, I can imagine work of breathing or clipped sentences or things like that could lend themselves to being interpreted by the AI listener.
Yeah. Interesting. That brings me to the note of, how much of, ‘Oh, it’s the front part of your leftshoulder.’ How much does that get translated into, the anterior, anterior medial, whatever part in the note, for the physician’s note versus – second question all at once – can the same process be leveraged to then print out and hand the patient a summary and say, here’s a summary of our visit today and have that more in lay language. Can you do that concurrently?
DR. PELO: Yes it can, it can do both of those things, but what won’t surprise you is every doctorwants their note to be slightly different. And so that’s part of a lot of the tooling we’re working on right now is, how do we make it so that each doctor can put their own voice, their own tone, how they wanttheir note to sound. But then also, like you said, as I hand this to my patient, we want them to know exactly what to do, when to do it, all of those things.
RAJ: And can it do that then?
DR. PELO: So yeah, you just click a button and it will rewrite all of those follow up instructions,when you’re supposed to see your primary care doctor, when you’re going to get your colonoscopy,whatever the different instructions are, who you need to call, and then what you should call me back about. And you’re able to hand that to the patient.
RAJ: And then is there the opportunity to leverage smart phrases, like ankle sprain follow up or, castcare info, and just say that, and have the AI hear it and
then say, okay, that means I’m going to drop this into the chart, either the medical chart or the patient’s version of the chart?
DR. PELO: Yes, those are the type of things that will be coming in the not distant future as peoplebeing able to use a lot of implicit instruction to the AI and have things automated for them like that.
RAJ: Super interesting. You mentioned that so far you’re using it mostly for American visits and thatyou’re churning it out to other countries and stuff. How do local idioms, and jargon, maybe not jargon, but, local phrases, if I say, something about, early innings, people in the States know what I’m talking about because baseball, but people in Australia are like, what are you talking about perhaps, I dunno? Is there a nuance there that needs to be addressed and is it being addressed?
DR. PELO: Yeah, absolutely. And the interesting thing is just take the U.S. just take Louisiana, New York, and Colorado or pick your three different locations around the U.S. and you start hearing those. But it is direct medical dictation.
And so now what you need is medical conversations and so that is something that we have collectedfor years and that we’ll continue to collect for years. And that will continue to get better at pulling out, what are those idioms? What do they mean? How do I interpret this into a medical note? Some of those things are a hundred percent solved. And some of those things are things that we’re still solving.
RAJ: Such an interesting time to be practicing right now. I can imagine some providers who are maybeafraid that adopting medical AI can somehow lead to either increased surveillance of their practice, especially, in the realm of, ‘you can’t order that test unless you get preauthorization’ or something like that. Is there a concern there?
DR. PELO: Now, I will say my personal belief, and this is a hundred percent just Jared’s opinion. Ithink there will come a time in the not too distant future and I’m going to say, five years, this is my gamble, that we will not have doctor-patient interactions, nurse-patient interactions non-recorded.
Like we will always have an AI helping us because we will learn that a human plus an AI is better medicine. Like it’s safer, it guides us, it helps us not miss things, it helps us not make mistakes. And so I do think that we are headed towards a time where pretty much every interaction inside a hospital, insidea
doctor’s office, is recorded if a patient opts into it so that they receive better, safer care.
RAJ: That’s exciting and terrifying at the same time.
DR. PELO: It is.
RAJ: Especially I think in the political climate of, we’re gonna politicize parts of healthcare and say you can’t do this. You can’t offer certain things. I think that’s going to be a real challenge.
Is ‘off the record’ type of thing – do those conversations still happen with DAX Copilot? Like where youturn it off and say, ‘Okay, now it’s just you and me, two humans in the room, nobody else’?
DR. PELO: Absolutely. Absolutely. You’ll have patients for sure who will say, ‘Hey, do you mindturning that off?’ And yeah, I think it’s important sometimes, right? For that patient to feel so safe that they can say things that they’ve been not wanting to tell anybody. They need that reassurance that it’s just you and them. And that happens even still with an AI Copilot.
RAJ: So as AI becomes more integrated, you said five years, probably everything’s going to be, you, me and the AI. What about the next 10? Let’s say 10 years from now, as AI becomes more integratedinto healthcare and perhaps it’s doing more than just listening and formulating notes. Where do you see the technology and where do you see DAX Copilot fitting in?
DR. PELO: Yeah, I think DAX Copilot is going to be used primarily for helping physicians practicemedicine better and helping nurses practice medicine better across the care team.
Where I think AI is going is, first, we’ll tackle administrative burden and really try to knock that out. And then I think we start getting into some interesting places and that is well, how do we use AI to help in clinical decision support?
How do we use AI to help even in guiding a patient through preoperative care or things like that. And we’re just beginning to understand that. And the FDA hasn’t really told us exactly how they’re going to regulate large language models. For the most part, it seems like they are putting the onus on thehealth system and the physician to validate that things are safe.
But I think over the next 10 years, what I’d love to see is as medical records become more and moreopen as we’re able to move our own medical records around and share our own medical records with AI.
Right now we are very focused on, how do we stop the loss of clinicians? How do we help with burnout? Eventually it’ll become, how do we help patients be healthier? How do we help patients in their home?
And so I think, we’re at this phase where it’s the early 1990s of the internet, where it was reallyexciting for a small group of people. And now the internet is just everywhere and we can’t imagine data not flowing freely and quickly.
And I think, 20 years from now, it’ll be the same with AI. We just can’t imagine not having this intelligence helping us all the time. And I think we’ll have that touching every part of our life. Andnothing is more intimate or more important than our own health. And so that’s where I think AI will eventually go.
RAJ: With the 15 million conversations or whatever that you’ve got, there might be biases, cause doctors have biases, right? There might be biases based on the data that the AI is trained on. Is there a way to ensure that that DAX Copilot avoids perpetuating those biases?
DR. PELO: Yeah. This is such an interesting topic, Raj, because of course, I want to just say no, there’s no bias. But like you and I know, like just practicing medicine in the U.S., we know that some of our minority groups are going to have worse outcomes just because that’s how medicine has been perpetuated in the U.S. And we have literally been recording medicine in the U.S. And so I can’t saythat 100 percent there is no bias in these models. But I would say what Microsoft is 100 percentcommitted to is reducing the amount of bias as much as possible. And I think that is the goal. How do we make these as fair and as equitable as possible so that we give better care.
And I can imagine a future where you can even fine tune these models. So they have a propensitytowards fairness and those are things that we will continue to work towards, is how do you make it so that we even nudge physicians away from bias? And so I think that’s an interesting place that we’ll seethese working as well.
RAJ: Yeah. I just read a study where they use different language about people who use drugs and they had listeners give their emotional responses to two different scenarios. And the scenarios were exactlythe same, except in one, the person was referred to as a drug abuser. And then in the other, the personwas
referred to as a person with a substance use disorder. And the amount of emotional difference -people rated their responses very differently depending on how that person was portrayed.
So little language stuff like that is what worries me, to be honest. Language is an approximation ofour thought and if we are handing over our language to an AI, I just worry that it is maybe taking a little bit of control or maybe not even control, but just making a difference, in the next person who reads my note and says, ‘Oh Raj must’ve been thinking this’, granted, if I don’t have time to chart, and if it’s busy, or if I’m staring at a computer while I’m trying to take that history and then trying to chart at the same time, then there are going to be a whole other group of problems.
So I appreciate that DAX Copilot is taking a lot of those off my plate. But I just worry about what technology in general might be replacing on my plate. But that’s maybe more philosophical than we need to get to.
If you could have our listeners walk away from this conversation, understanding one key thing about DAX Copilot, what would it be?
DR. PELO: I would say… we share stories all the time around the great outcomes, and I heard this story the other day where a physician came home, and comes in and says hi to his wife and his kids.And his wife is like, ‘What are you doing here? Like, you’re two hours early.’ And he said, ‘Ah, DAX Copilot wrote my notes. I’m home. I don’t, I didn’t have to stay late today.’
And I think the number one thing that I worry about in the U. S. is that we will continue to lose physicians who our patients need. And so if you’re a physician who’s lost hope and you thought, ‘I’m burned out, I don’t want to keep doing this.’ Then I want you to know two things: one, what you domatters every single day. It matters to the patients you serve. And then two, we’re building tools to make it so that you are able to do your work and do it with more joy. And that’s really what I want people to take away, that there’s hope for the future.
RAJ: Thanks to Dr. Pelo for speaking with us.
This is DDX, a podcast by Figure 1. Figure 1 is an app that lets doctors share clinical images and knowledge about difficult-to-diagnose cases.
I’m Dr. Raj Bhardwaj, host and story editor of DDX.
Head over to figure1.com/ddx where you can find full show notes, speaker bios and photos.
This season of DDX was produced in partnership with and sponsored by Microsoft.
Thanks for listening!t for speaking with us.
This is DDX, a podcast by Figure 1. Figure 1 is an app that lets doctors share clinical images and knowledge about difficult-to-diagnose cases.
I’m Dr. Raj Bhardwaj, host and story editor of DDX.
Head over to figure1.com/ddx where you can find full show notes, speaker bios and photos.
This season of DDX was produced in partnership with and sponsored by Microsoft.
Thanks for listening!