Walk into any small medical practice today and ask the front-desk staff if they’ve ever pasted a chart note into ChatGPT to “rewrite this so the patient understands it” or to “summarize this lab result.” A lot of them will say yes. Some will say no but their browser history says otherwise. A few will look genuinely surprised that anyone’s asking.
Here’s what they’re not thinking about: protected health information (PHI) under HIPAA includes more than the obvious identifiers. It includes anything that, in combination, could identify a patient — symptoms plus visit date, lab values plus condition, even free-text descriptions if specific enough. Once that text leaves the practice’s network and lands in a cloud AI service, the practice has technically engaged that AI vendor as a business associate, and a Business Associate Agreement (BAA) is required. The big AI providers offer BAAs only on enterprise tiers — usually $$$$ a month. Most practices using ChatGPT or Claude on patient data have no BAA in place at all.
That’s a HIPAA breach waiting to be discovered. And it’s already everywhere.
Why this is suddenly a real problem
For years, the assumption was that nobody on staff would use a “ChatGPT” on actual patient data — it’d be obvious that PHI shouldn’t go to a third-party server. That assumption no longer holds. The tools are too useful. Practice managers, MAs, billers, NPs, even physicians are routinely pasting things in: prior-auth letters, patient instructions, summary letters to referring providers, insurance appeal templates, lab interpretations.
Each of those sessions, on the major cloud AI providers, is potentially a HIPAA-reportable event. The practice doesn’t know it. The vendor doesn’t know who the patient is. But under the regulation, the disclosure happened the moment the text crossed the network boundary without a BAA in place.
OCR enforcement actions in the past few years have been heavy on exactly this kind of “we didn’t realize we were using a third-party processor” finding. Penalties for unintentional disclosure under the HIPAA Privacy Rule start at $137 per violation and can reach $68,928 per violation depending on culpability — and “violations” can be counted per record disclosed. A single staff member pasting 30 patient summaries into ChatGPT over a quarter is, by the regulation’s math, 30 violations.
Most practices are not okay if they’re audited tomorrow.
The fix that actually works: on-device AI
The practice doesn’t have to give up AI. It just has to keep it on the practice’s own machines, where there’s no third-party processor relationship.
Modern Apple Silicon Macs — the kind a lot of practices already have at the front desk or in clinician offices — can run open-weight language models locally. The model runs in the Mac’s unified memory using Apple’s MLX framework. Prompts and responses never touch a network connection. There’s no API key, no vendor account, no outbound traffic to log or audit.
For HIPAA purposes, this changes the legal posture entirely. Software running on a covered entity’s own hardware, with no data transmission outside the entity’s secured environment, is not a third-party disclosure. It’s the same legal category as a Word document — local software processing data the practice already has lawful access to.
The HIPAA Security Rule still applies (the Mac itself needs to be physically secured, encrypted at rest, with access controls), but those are the same controls the practice already runs for its EMR workstation. No new vendor risk. No new BAA. No quarterly compliance review of the AI provider’s SOC 2 report.
What it looks like inside a practice
A typical small-practice install:
- 2-5 MacBooks (front desk, clinician workstations, billing). Most practices already have these.
- An open-source MLX server installed on each, running a 31B or 70B language model.
- A simple chat interface on the desktop. Looks and feels like ChatGPT. Behaves the same. Just doesn’t phone anywhere.
- A one-page HIPAA AI Use Policy documenting that the practice’s AI tools run on-premises with no third-party data processors. This goes in the practice’s compliance binder.
- An hour of staff training on what tasks make sense for the local AI vs. what should still go through the EMR.
After install, the practice’s AI usage is HIPAA-clean. Nothing to add to the BAA log. Nothing to disclose to patients. Nothing to argue about in an audit.
The specific wins for a medical practice
- Patient instructions in plain language. Convert “post-op care: keep wound site dry x 5 days, rotate dressing q12h, NSAIDs prn” into a paragraph the patient will actually read. Local model, no PHI exposure.
- Prior auth letters. Drafting these from chart notes is a huge time sink. Local AI can generate the first draft from the relevant note, with the chart never leaving the practice.
- Insurance appeals. Same pattern. The AI sees the denial letter and the relevant clinical history; the practice’s data stays local.
- Letters to referring providers. Clean, professional, fast — without sending the patient’s chart to a cloud LLM.
- Patient education content customized to the practice (not the same generic handouts every other clinic uses).
None of these are dramatic. All of them are time savers worth tens of hours per month per provider. And every one of them is safe to do with on-device AI in a way that’s genuinely not safe to do with cloud AI.
What the cost actually looks like
A small practice using cloud AI properly (with a BAA-covered enterprise tier) is looking at $40-100 per user per month, plus the legal and compliance overhead of vetting the vendor and adding them to the BAA log. That’s $5,000-15,000+ per year in subscription cost for a 5-person practice, before any compliance staff time.
A one-time on-device install for the same practice runs $8,000 to $15,000 all-in (hardware aside — most practices already have the Macs). After that: zero recurring AI subscription cost. The AI runs on hardware the practice already owns, indefinitely.
The financial argument is real, but it’s secondary to the compliance argument. The compliance argument is: on-device AI is the only AI configuration that doesn’t create a HIPAA business-associate relationship. That’s not a marginal advantage. That’s a categorical difference.
Who should be looking at this now
- Solo and small group practices doing primary care, behavioral health, dermatology, OB/GYN, mental health, dentistry — anywhere clinicians are tempted to use AI on chart text.
- Therapy and counseling practices where session notes are particularly sensitive and where most cloud AI tools are an obvious non-starter.
- Concierge / direct-pay practices where patients explicitly chose the practice for higher privacy expectations than chain medicine offers.
- Practices that already had a HIPAA scare — a near-miss, an OCR letter, a malware incident — where the leadership now takes data flow questions seriously.
- Any practice in California, New York, Massachusetts, or other states with privacy laws that exceed HIPAA in scope.
If the practice’s leadership doesn’t know exactly what AI tools the staff are currently using on patient text, that’s the answer to “should we look at this.” The fix is not to ban AI (it’ll go underground), it’s to give the practice an AI that doesn’t create a vendor-risk problem.
I do on-device AI installations for small medical and therapy practices — fixed-fee, one week start to finish, including the HIPAA AI Use Policy and staff training. If your practice is quietly accumulating AI usage without a clear compliance posture, this is the cleanest fix on the market.
More detail: AirGap AI — book a 15-minute call from that page and I’ll walk through whether on-device is the right fit for your specific setup.
— Matt Macosko, Nice Dreamz LLC
The open-source software the install is built on is public at github.com/nicedreamzapp/claude-code-local — you or your IT contractor can review exactly what runs on practice hardware.