Asking ChatGPT without Your Full Health History Is Guessing
If AI doesn't have your health history, it answers anyway. That's dangerous.
This is the second in a series on using AI for personal health.
Full Disclosure: I built HealthScout, an app to help patients navigate our complex healthcare system.
In an Oxford AI health study, only 35% of the people asking health questions got the right answer. Not great.
In the same study, when the AI model was given clean, structured cases, it scored 95%. The main difference was context.
Much of the press generated from the study took the angle that AI doesn’t work for health. But that 60-point accuracy drop wasn’t because the AI didn’t work. The AI models were identical.
What changed was what the AI had to work with. Researchers gave the AI complete, organized information. Real patients couldn’t do that. They didn’t know what was relevant, and they didn’t have their health records handy.
My Labs Looked Normal. They Weren’t.
I saw the Oxford study's findings about health context play out in my own life before the study was even published.
During an annual physical, I complained to my doctor about my “old man pains” (knee, back, shoulder) that increasingly affected my ability to be active. The doctor recommended I try an over-the-counter NSAID (non-steroidal anti-inflammatory drug) called naproxen.
It was magical. After a few days taking it twice a day (as he suggested), I felt amazing. It worked so well that I eventually switched to a prescription NSAID called celecoxib, which studies show is easier on the stomach. I thought I'd done my homework.
A year later, at my next wellness check, my eGFR came back at 69.
My doctor scanned the results, saw nothing flagged as “High,” and moved on. I did too. With 30 minutes for a wellness check, you cover a lot of ground quickly.
Four months later, I started building HealthScout. My own health records were the only data I had to test with. As soon as a working version was running, I started asking it questions. One of my early test questions was:
“Do you see any trends that show a relationship between my medications and my lab results?”
To my surprise, HealthScout suggested that my “recent kidney function decline” could have something to do with my regular use of NSAIDs.
What recent kidney function decline? I had no idea what it was talking about.
As it turns out, my primary care doctor only had records from One Medical. HealthScout had records from three providers spanning over a decade.
With that full history, it could see that my steady-state eGFR was around 90. A drop to 69 in seven months isn’t “low-normal.” It’s a 30% decline, which with full context is at least a yellow flag. HealthScout also knew that while most primary care physicians use the standard eGFR threshold of 60 as the cutoff for concern, nephrologists use a more granular classification system where 69 is already approaching moderate kidney disease.
No single provider would have seen any of this. Each one only had their own slice of my history.
My eGFR dropped all the way to 50 on my next test, low enough that I was diagnosed with Chronic Kidney Disease Stage 3a. A biopsy confirmed the finding: the NSAIDs had caused the decline. The good news: I’m off them now, and my kidneys are slowly recovering.
The Context Gap Is Structural
My doctor is a good guy who was put into a difficult situation — not enough data and not enough time. The Oxford study confirmed what I experienced firsthand: more context produced better answers. Less of it produced dangerous ones.
HealthScout connects to over 10,000 healthcare systems through Apple Health, including Labcorp and Quest Diagnostics, where most Americans get their blood drawn. It also accepts uploaded documents: photos of lab printouts, PDFs of clinical notes, or even pictures of reports snapped from your computer screen. The AI works from your complete history, across providers, going back as far as your records exist.
Your actual health history exists. It's just in multiple systems that don't talk to each other. Do yourself a favor and put it all in one place.
HealthScout is available now on the App Store. No email required, no account needed. Connect your providers, upload a record, and ask your first question in under a minute.