Context
Chart abstraction — turning free-text clinical notes into structured research fields — is one of the most valuable and least glamorous jobs in health data. Done by hand it is slow and costly. Done carelessly with a cloud LLM it is a privacy incident. At T-CAIREM, our principal is building the third option.
The problem
The charts contain PHI throughout, so the usual answer — send text to a frontier-model API — is off the table. Whatever does the abstraction has to run on hardware the hospital controls, which caps model size and forces engineering discipline that API users never face.
What we built
Instead of one giant prompt, the pipeline is a directed acyclic graph: each of the 256 target clinical fields is resolved by a node with its own context, extraction logic, and validation. Failures are localized, auditable, and re-runnable — properties reviewers actually ask about.
The models are deliberately small — under 30B parameters — evaluated systematically and fine-tuned on medical data (Gemma among them) so that ontology grounding comes from training, not from model scale. The whole system is containerized and runs on two H100s on-premises.
Key decisions
Small models over API giants: the accuracy gap closes fast when a model is fine-tuned on the target ontology and each DAG node only has one narrow job. And a sub-30B model fits on hospital-owned GPUs, which is the entire point.
Decomposition over end-to-end magic: 256 fields through one prompt is unauditable. 256 nodes with explicit dependencies can be validated field-by-field against grounded chart data — the difference between a demo and a system a compliance office will sign.
Outcome
A containerized, on-prem abstraction system extracting 256 clinical fields under continuous evaluation, with fine-tuning work ongoing. The pattern — small fine-tuned models, decomposed pipelines, hardware inside the fence — is now our default recommendation for any PHI-adjacent AI workload.