Generic AI Recommended a Drug I Can't Take
It didn't ask about my health history. It recommended a drug that would damage my kidneys.
This is the first in a series on using AI for personal health.
Full Disclosure: I built HealthScout, an app to help patients navigate our complex healthcare system.
I typed the same question into two AI apps: “I woke up with a headache. What should I do?”
ChatGPT went straight to a comprehensive six-step response. Drink water. Eat something light. Take a pain reliever. Stretch your neck. Get some fresh air. Try coffee.
HealthScout presented nine numbered scenarios and asked me to pick the ones that fit.
ChatGPT Suggested Taking a Drug That Would Hurt Me
ChatGPT’s third recommendation, right under “Take a Pain Reliever (if needed),” is ibuprofen.
I have chronic kidney disease. Ibuprofen accelerates kidney damage. It’s one of the first things a nephrologist tells you to avoid.
ChatGPT didn’t know I had CKD. It didn’t ask. So it handed me advice a doctor familiar with my history would never give.
This isn’t rare. The person asking about a headache might be on blood thinners, might be pregnant, or might be taking a medication that interacts badly with what’s being recommended. Generic AI can’t know any of that, because it doesn’t ask, and it doesn’t have your records.
What HealthScout Did Instead
Before answering anything, HealthScout gave me nine numbered scenarios and asked me to pick the ones that fit.
Each option covered a different angle: severity, context, medication status, specific health concern. I typed “1 and 8”, and it had what it needed.
Look at what option 8 actually said: “I’m concerned about what pain relief I can take because of my chronic kidney disease.”
HealthScout already knew I had CKD. It surfaced that scenario specifically because it was in my records. I didn’t explain my history. I just confirmed it.
Why Numbered Options Matter
Describing your headache in words takes effort, and most people don’t do it well. You leave things out. You don’t know what’s relevant.
Numbered scenarios covering distinct angles are faster and more accurate. You pick one or several, and the app has what it needs to give you something useful.
ChatGPT does eventually ask for clarification. If you scroll past the six steps, past the warning signs, past the “quick routine,” there’s a section at the very bottom: “if you want, tell me where the headache is and how strong it is.”
After the laundry list of advice. After the ibuprofen recommendation.
Then HealthScout Answered About Me
Once I replied with “1 and 8,” HealthScout pulled from what it already knew: my CKD diagnosis, my current medications, and my history. The response was about my headache, filtered through my health picture.
For a lot of questions, a wall of text is exactly what you want. For research or planning questions, a broad answer is fine. For health questions, it isn’t.
That’s what purpose-built means in practice. A system designed to understand your situation before drawing a conclusion, because the right answer for most people may be the wrong answer for you.
HealthScout is available now on the App Store. No email required, no account. Upload one record and ask your first question in under a minute.