Dr. Karandeep Singh, recruited from University of Michigan, will guide integration of machine learning into day-to-day medical practice
UC San Diego Health is importing a sharp mind to help guide its efforts in artificial intelligence.
The university health system recently announced the recruitment of Dr. Karandeep Singh to serve as its first “Chief Health AI Officer,” appointing the University of Michigan researcher and clinician to fill the Joan and Irwin Jacobs Endowed Chair in Digital Health Innovation at the UCSD School of Medicine.
Singh, states Dr. John M. Carethers, vice chancellor for Health Sciences at UCSD in a recent university announcement, is known for his expertise in this rapidly growing field.
“Having worked with Dr. Singh previously in our careers, I have witnessed his impressive track record and deep expertise in health care technology firsthand,” Carethers said. “He has become a national leader in the evaluation of proprietary AI technologies, a specialist in AI governance and has actively engaged in influencing AI policy with considerable experience and impact.”
That work includes a paper in the New England Journal of Medicine this year on developing a method to precisely assess the net benefit of machine learning models when health care resources are constrained. A piece last year in Nature Machine Intelligence covered using such technology to predict acute kidney injury across health systems, and Singh also participated in a 2021 article in the Journal of the American Medical Association that found deficiencies in a model said to be able to predict sepsis, a deadly condition that can cause organ-damaging inflammation.
A nephrologist trained at the University of Michigan who completed his internal medicine residency at UCLA Medical Center, Singh also earned a master’s degree in biomedical informatics from Harvard Medical School. At Michigan, he served as the university’s chief medical information officer of artificial intelligence.
With his new role effective last week, Singh said he plans to move his family to San Diego. His wife, Dr. Tejpreet Kaur Nakai, a primary care and internal medicine specialist, also plans to practice in San Diego. The couple has two children, ages 7 and 10.
UC San Diego Health is no stranger to using new AI technology in its operations, deploying software in its emergency departments to help predict which patients might develop sepsis. A pilot program with Epic Systems, maker of the nation’s most popular electronic records system, is already using ChatGPT AI systems to draft routine emails to patients. The system helps doctors save time with correspondence while maintaining oversight of their messaging.
UCSD is investing heavily in the infrastructure to make itself among the leading locations for research into how AI can increase the efficiency and accuracy of health care delivery, and a big part of that push will be a high-tech hub set up to collect and collate disparate data sources generated at the bedside and in clinics. That effort is enhanced by a $22 million grant from the Joan and Irwin Jacobs Center for Health Innovation on campus.
Singh’s role will be twofold: helping guide the oversight and integration of already vetted AI technology into UCSD operations and conducting research into the safety and effectiveness of newly developed models and methods.
UCSD is the third University of California campus to add a Chief Health AI Officer position, following UC San Francisco Health and UC Davis Health, which made similar hires in 2023.
Singh recently took some time to discuss his approach to AI in health care with The San Diego Union-Tribune. What follows is an edited question-and-answer session with the new executive:
Q: Your work has recently been cited in national policy around AI in health care. Given that this city is about as far from Washington as one can get and still be in the United States, why UC San Diego?
A: The University of California has a big footprint in D.C. (Dr.) Chris Longhurst (UCSD chief medical and digital officer) has been in D.C. on this topic more in the last year than I have, so I think there is actually plenty of opportunity to participate in policy discussions. But, while policy is an important part of our mission, the primary goal, I think, is taking care of the community. And that’s what drew me to UCSD, the opportunity to really think big about how we use AI to make both receiving and delivering care a better experience.
Q: What would you say to someone who asserted that this type of position is something of an extravagance in the current climate of worker shortages that have permeated health care since the COVID-19 pandemic?
A: When people first hear about it they’re like, “wait, why do we need someone in a leadership role for AI?” I think there are two reasons why this position needs to exist. The first is that you need someone in health systems who’s knowledgeable and accountable for how this technology is used. The second is really about planning and strategy, needing to have someone in the room when decisions are being made about how we organize our care, who is up to speed with the AI tools that are available, what’s coming and what might we want to consider using.
Q: With the sudden arrival of generative AI, especially as embodied by ChatGPT, it seems that there is a sort of digital gold rush mentality going on at the moment with new products announced daily. How does an AI director recognize true promise among so much hype?
A: A lot of our work which has been cited at the federal level has involved trying to evaluate systems and make sure they actually work as well as they’re supposed to. I hope to bring some of that rigor to this job so that we’re not solely relying on what’s being told to us by companies but that we’re actually going to verify those claims through our own research and investigation.
I come from a place of optimism but also skepticism about any one tool. I think a lot of what I’ll be doing in my role is going to be trying to help us cut through the clutter, rigorously looking at what’s out there so we can be leaders in helping others identify what works and what doesn’t.
Q: Recent guidance from the Biden administration says that AI should “lead to health care outcomes that are fair, appropriate, valid, effective and safe.” Given that even their creators would say that these AI models are black boxes, no one perfectly understands their inner workings, how do you make sure that these goals, especially safety and accuracy, are upheld?
A: Even if it is a black box, I think we can evaluate any system. We can run it silently, meaning that we just hook it up to see how well it does with real-world information, but we don’t actually make any decisions based on what it finds. By comparing real-world results with what the model finds, we can validate if it’s a good system or not.
Q: Increasing efficiency is one of the biggest promises of integrating machine learning and artificial intelligence models into day-to-day health care. But, in the early going, we’ve already seen that some models can end up creating more work. For example, systems used to evaluate chest X‑rays, while able to find anomalies that humans might miss, also have been shown to kick up many more false positives. Is there a danger that some uses of AI will require so much human oversight that they will not meaningfully increase efficiency?
A: Absolutely, and I think you’ve got to look at things in the way you intend to use them. The same tool used in two different workflows might have very different effects. A radiologist is not going to be happy having to stop their work and react to an algorithm that’s flagging abnormalities that aren’t that abnormal. That’s just making more work. But what if, instead of interrupting the radiologists, instead we’re using the algorithm to order their work so that the next image they get to review is the one that’s most likely to have an abnormality? They’re still reading all of the same images, but they might just be reading them in a more efficient order.
Q: Given your work understanding and validating AI technology, what is one area of health care that you think is on the cusp of change, and what is one area where you feel like we’re still a long ways away from ever being able to use AI to influence patient care?
A: Large language and multimodal models that respond with text explanations are probably what anyone would say are on the cusp. Obviously, using these models to draft written replies to patients has already been piloted at UCSD, and summarizing information is another one that’s being tested right now. Let’s say someone is ready to go home, and they’ve been in the hospital for a month. It can be a lot of work to accurately pull together, sort of in a nutshell, everything that happened to this person during that time. And that can also be a challenge when a patient is handed off from one doctor to another. So accurate summarization to reduce the documentation burden is another idea that’s coming soon. Ambient documentation is another one that is right on the cusp. That’s where, instead of clinicians taking notes while staring at a computer screen, you can just talk to the patient naturally and the encounter is being processed in real time, not just pure transcription but also formatting that information in a way that the clinician needs to do in order to document that encounter. That kind of technology is being tested now in a few places, but we haven’t yet seen the big study that tells us whether it works or not. I think automated decision making is very far away. There is just so much that clinicians do that isn’t like math. They’re trying to understand that person in front of them, and what is that person’s value system? What’s important to them? What information or advice can you give them to help them make the best possible decision for themselves? A lot of what AI is about in health care today is what (chief AI officer) Sara Murray at UCSF calls “keyboard liberation.” It’s about getting us away from keyboards and letting us spend more time with patients.
This article was originally published in the San Diego Union-Tribune.
