Clinical Text Simplifier · LoRA Fine-Tuning Case Study
Simpler Isn't Always Safer
Can an open, 0.5B-parameter LLM be LoRA-tuned on a CPU to simplify biomedical text better than the same model prompted zero-shot? Scored against PLABA — NIH/NLM's real, professionally human-simplified dataset — not asserted.
Clinical writing is dense on purpose — can a tiny model fix that?
Large frontier models can simplify biomedical text reasonably well out of the box. That isn't the interesting question for a portfolio project — the narrower, falsifiable one is: can a small (~0.5B parameter) open model, fine-tuned with LoRA on a CPU in a modest amount of wall-clock time, get measurably better at this one task than the same base model prompted zero-shot?
Scored against PLABA — 75 consumer-health topics, 749 PubMed abstracts, every sentence adapted into plain language by professional NLM annotators. Real human references, not self-graded output.
CPU-only, validated before committing
This ran on a 4-core laptop CPU, no GPU — the same class of hardware that caused an out-of-memory crash on a past project. A smoke test came back clean, but at 8.8s/example, three epochs over the full 6,202-example split would have taken ~45 hours.
Qwen2.5-0.5B-Instruct
Apache 2.0, 496M params including the LM head.
LoRA, rank 16
2.16M trainable params — 0.44% of the full model. Base weights never change.
batch_size=1 fastest
Larger batches and extra threads were both slower on this CPU — benchmarked, not assumed.
800 examples, 1 epoch
~70–90 minutes — a documented scope reduction, not a hidden one.
Every metric moved the same direction
200 held-out test sentences, identical greedy decoding for both models, scored against the same human references.
Readability moved further too: fine-tuned outputs average Flesch-Kincaid grade 11.08 vs. 12.24 zero-shot, against source sentences averaging grade 14.6.
Reading the wins
- Source
- Currently, only one half of the people who have diabetes mellitus have been diagnosed.
- Reference
- Currently, only half of those with diabetes mellitus have been identified.
- Baseline
- Currently, less than half of individuals suffering from type 2 diabetes have been identified as having the condition.
- Fine-tuned
- Only half of those with diabetes have been diagnosed.
- Source
- These changes provide an easier and more reliable means of diagnosing persons at risk of complications from hyperglycemia.
- Reference
- These changes help improve identification of those at risk from effects of high blood sugar.
- Baseline
- These adjustments make it easier to identify people who might have problems with high blood sugar earlier on.
- Fine-tuned
- These changes make it easier to diagnose people who may have problems with high blood sugar.
An honest failure
The fine-tune helped on every metric — that isn't the same claim as "safe for real patient-facing text." Reported directly rather than smoothed over:
- Source
- It is clear that these changes in diagnostic criteria have major importance both for individuals and for resource planning at a national level.
- Reference
- It is clear that these changes in identification criteria have major importance both for individuals and for planning at a national level.
- Baseline
- These changes in diagnostic criteria are crucial for both individual patients and overall resource management at a national scale.
- Fine-tuned
- It's important to change how doctors diagnose patients so they can better treat them.
What surprised us
Given how little supervision 800 examples/1 epoch represents, a split result across metrics seemed more likely going in — better BLEU/ROUGE from picking up PLABA's vocabulary, but no real readability gain, or vice versa. Instead every metric moved the same direction, by a consistent margin.
The fine-tuned model also simplifies more aggressively than the human annotators do by grade level (−3.51 grades vs. source, versus the human references' own −1.40) — and that isn't a free win. Some of that extra simplification is genuinely tighter phrasing; some of it is the model dropping real content for vaguer language that still scores well on n-gram overlap.
Limitations
The official TREC-2024 test split sits behind NIST registration that couldn't be completed non-interactively — this project uses a self-constructed topic-level split instead, so numbers aren't directly comparable to the published leaderboard. A single 0.5B model, one LoRA configuration, no hyperparameter sweep. Automatic metrics only — no clinician or lay-reader panel reviewed outputs. Evaluation used 200 of 1,242 test sentences, a documented scope reduction for CPU wall-clock time.