As artificial intelligence (AI) makes medical knowledge increasingly easy to retrieve, health-care education may need to place greater emphasis on the abilities that cannot be reduced to information recall: reasoning, applying knowledge in new contexts, critically evaluating AI, and using AI in ways that support continued learning and collaboration.
That was a central theme of a discussion at Cleveland Clinic’s A.I. Summit for Healthcare Professionals. The session, moderated by James Stoller, MD, MS, Chairman of the Education Institute at Cleveland Clinic, featured Lisa Christensen, Global Head of Learning Design and Innovation at McKinsey & Company, and Travis Zack, MD, PhD, Chief Medical Officer at OpenEvidence and an oncologist at the University of California, San Francisco.
The panelists discussed how clinicians should learn to work with AI and how medical educators can prepare them to do so without allowing the technology to displace the critical thinking they need to develop and maintain.
From memorization to transfer
Because AI has made access to information easier, “there may be more and more of a shift toward making sure that the [human] reasoning component is fundamentally solid," Dr. Zack suggested. "That said, I still think that to reason, even if you have the information present, you need to have some amount of fundamental underpinning understanding of how things work.”
He added, “It is important to learn how to reason, both with and without the tools that are available.”
As she considered how learning is changing, Ms. Christensen agreed, emphasizing that learners may need to memorize information differently than they once did. She advocated for a “teach for transfer” approach that helps learners apply what they know in new situations, recognize similarities and differences across contexts, and strengthen analogical reasoning.
Domain knowledge and the purpose for which AI is being used also shapes how its outputs should be critically evaluated, Dr. Zack suggested. A nurse, resident, attending hospitalist, and subspecialist may all ask the same question but have different goals and therefore need different knowledge to evaluate the response.
Ms. Christensen made a similar point: “For those different kinds of practitioners, to what end are they partnering with these tools, and what is it that [they’re] trying to achieve? That is where you start to find some of the nuances of the different ways that people might need to learn to use the technology.”
Preventing 'never skilling'
Alongside the risk of clinicians losing established abilities due to increasing reliance on AI, or "deskilling," the panelists raised another concern: New learners may never develop those abilities in the first place if they jump right into using AI.
Ms. Christensen cautioned that greater productivity is not necessarily indicative of greater knowledge. In her organization, she said, AI can make junior colleagues “10 times faster,” while also degrading the quality of their thinking if they do not challenge, pressure-test, explain, and defend the output. She described seeing highly AI-forward junior colleagues produce work they could not fully explain because they had not performed the underlying thinking.
To counter this “never skilling” and reinforce cognition, her teams have experimented with varying the sequence of learning activities. For example, people may first develop a problem statement and then use AI to expand it or begin with an AI-generated draft and identify where it is wrong or could be improved. The goal, she said, is to develop the ability to think in both directions: “tool first, brain first.”
Dr. Zack similarly compared the distinction between passive and active AI use with navigating by GPS. A person can repeatedly follow directions without learning the route or can pay attention to the recommended path and gradually build a richer understanding of the area and alternate routes to the destination. The same principle applies to clinical AI applications such as scribes and decision-support tools, he said: active engagement can turn routine use into an opportunity to learn.
“You’ve now been exposed to many more things than you would have if you had just gone the single [route with GPS],” he stated. “It’s an educator’s role to train someone not just to learn how to use the tool for safety of the current patient, but [also] for continuing education, to actually learn to use the tool for the safety of future patients.”
Designing tools that develop learners
“The way that the tool supports you is going to make a difference in how you think and whether or not you grow in your skills by doing the work,” Ms. Christensen said. An AI system could ask users to identify the most important insights or use Socratic questions to broaden their thinking, for example. "We want it to nudge you. We want it to push you. We want it to ask you some questions."
She described a question her team is exploring: “How do we develop plugins that go inside these proprietary tools to put guardrails around the experience so that it is also [acting] in service of your development?”
Dr. Zack framed the development of tools that support learning around the “unknown unknowns,” or information users may not realize they need to provide or consider when asking a question or evaluating an answer. One potential feature, he suggested, would be helping users identify what they may have missed so they are less likely to miss it the next time, such as if a physician forgot to mention a specific mutation or performance set for greater specificity.
Data generated through AI use may also inform learning over time. Dr. Zack pointed to search history and self-assessment or -summarization mechanisms that could help users review what they have encountered over time, including through spaced repetition. Ms. Christensen similarly discussed examining how people use the technology and identifying patterns in those interactions that could guide efforts to teach them more effectively.
Dr. Zack cautioned, however, that surveillance would need to be done responsibly and might affect how clinicians use an AI system. “I think there’s a balance and a challenge there,” he said.
The panelists also discussed tool fluency, which Ms. Christensen defined as “how successfully you’re able to partner with a tool … and a base understanding of the way that the tool is likely to approach and interact with you.” General-purpose large language models may flatter users, reinforce their views, and give them confidence in information that may be incorrect, she noted, but “knowledge of that capability is not enough to offset the outcome.” Her organization is designing a set of agentic skills to work within existing tools, challenging rather than praising the user.
In his own approach to tool development, Dr. Zack said, the focus is on the evidence and papers that answer the question, rather than on the user’s opinion. Meanwhile, he suggested that tool design is only part of the equation; educators also need to “dive deep” to provide learners with a foundational understanding of how AI systems are built and work, and where their limitations may arise.
Preserving human learning
After much of the discussion focused on how AI should be used and designed to support learning in the health-care sector, Ms. Christensen turned to the role of human relationships. She described her organization’s emphasis on apprenticeship and the ongoing obligation for colleagues to teach one another and help one another grow.
“One of the things that makes me feel very hopeful is that we still exist in a world where we get to and need to work with other human beings. That relational component of learning is so important,” she said. “That’s a whole part of this eventual ecosystem of developing in an AI era where that’s going to continue to be important.”
Looking 5 years ahead, she characterized the most important change AI may bring to medical education as “opportunity,” including capabilities that cannot yet be envisioned but can still be shaped intentionally and around values. That opportunity, Dr. Zack suggested, could include strengthening connections across disciplines. As AI raises the knowledge base available to everyone from patients to health system leaders, he said, it may create more opportunities for collaboration and knowledge transfer.
