#99. Context Readiness

For the last three years, most of the argument about clinical AI has been about knowledge. Does the model know enough medicine to get the diagnosis right? Can it pass the exam? Can it beat a doctor?
A new article in The Lancet Digital Health argues that question is close to settled, and that we have been testing the wrong thing. Its title summarises it well: “Context is built, not given”.
The question has moved
The authors start by conceding the point most sceptics have been defending. A 2026 Science study found that on physician-level reasoning tasks, and on real emergency department cases where a second opinion was sought, “a frontier large language model has outperformed physician baselines.”
So the knowledge question is “no longer speculative”. The harder one, they write, is “what happens when AI is given a patient’s medical chart to review.”
That is not a hypothetical either. They point to three products announced this year that do exactly that: OpenEvidence’s work with Cedars-Sinai on “patient-aware” clinical AI, Abridge’s “context-aware clinical intelligence” rolled out to its partner health systems, and OpenAI’s move to let healthcare organisations connect EHR data to ChatGPT. All three are sold on being able to pull the relevant patient context out of the record.
Building context is not judging it
The industry already has a name for the work of getting the right information in front of a model: context engineering. The authors describe it as “filling the context window with just the right amount of information for an AI system to identify the best next step.” In clinical AI that means selecting, retrieving, structuring and updating information from records, tools, guidelines and local resources.
Their point is that this isn’t enough. “Building context is not the same as judging whether it is enough.” Give a model the whole unfiltered chart and it may struggle to work out when the symptoms began. Give it a tidy summary and the summary “might omit an examination finding that could change the differential diagnosis.”
So they propose a new term. Context readiness is “the ability to evaluate whether the available context is sufficient and reliable for the decision at hand and to identify what is missing, outdated, contradictory, or unverified.”
The examples given will be familiar to anyone who has read a real patient record. An AI system should be able to recognise when:
- a medication list conflicts with a recent note,
- a lab value is too old to rely on, or
- a summary diagnosis has no examination findings to support it.
And having spotted the problem, it should do something sensible about it: answer, ask for more information, retrieve more context, check a source, or “defer the decision until the context is safer.”
Context readiness “is a property of the deployed system, not of the context window alone.” And it depends on “what is retrieved, when it was recorded, where it came from, and whether the sources agree.”
In other words, it isn’t just about the model. It’s about the model, the integration and the record, together.
What our benchmarks don’t measure
According to the authors, “most current evaluations do not assess whether an AI system can identify such deficiencies in the available context and respond appropriately.” They test how well a model answers from the information it is handed, and stop there. “Such evaluation is not sufficient for clinical safety.”
They cite two pieces of evidence for why that gap is real:
- Models are sensitive to what they’re given. A 2024 Nature Medicine study found large language models are affected by the amount and order of the information they receive, and that fitting them into real workflows is “not straightforward”.
- A strong model doesn’t make a strong team. One of the author’s own randomised trial in JAMA found that giving physicians access to an LLM did not necessarily improve their diagnostic reasoning compared with usual resources. As the article says: “A strong model does not automatically make clinical AI safe in practice.”
To show the gap, they draw a simple two-by-two map. One axis asks whether the AI gets a fixed set of information at the start, or can gather and update it as care unfolds. The other asks whether we are testing the quality of the answer, or the AI’s judgement about whether the context is good enough.

Figure: Shimizu and Goh, The Lancet Digital Health, 2026 (CC BY 4.0).
Most published evaluation sits in the bottom left. The products now being sold operate in the top right. The authors are clear the quadrants are “heuristic regions on a continuous map”, and that the issue “is not case complexity, but whether evaluation stops at the answer or also asks whether enough information was available to answer safely.”
Why this matters
A patient’s context is rarely in one place. It is split across a GP record, one or more hospital systems, a shared care record, scanned letters, and whatever the patient tells you on the day. Different systems record the same medicine, allergy or problem in different ways and at different times.
That makes “where it came from, and whether the sources agree” the central question. An AI that answers fluently from whichever source it happened to retrieve first is a different risk from one that notices two sources disagree and says so.
AI scribes are now writing a growing share of the notes in our records. Tomorrow’s decision-support tools will read those notes as context. If we don’t evaluate whether AI can judge the quality of what it’s reading, we’re building a loop where one AI writes the record and another trusts it
The article closes with a line worth considering: “The safest clinical AI might not be the model that always answers from the context it is given, but the one that recognises when the context is not yet fit for decision.”
#21stCenturyMedicine #TheDataAndDigitalEngine #WorkforceReset