AMD Developer Hackathon · 2026

Path to Care

Multimodal, agentic triage decision-support for rural healthcare in the Global South. A phone photo plus a typed narrative becomes a top-3 condition guess, a Red / Yellow / Green urgency, and a structured pre-visit SOAP for the clinic doctor — contextualized by village distance, cost, and harvest season. It never diagnoses.

Live demo on Hugging Face GitHub LoRA adapter

What it does

A community health worker (or the patient) sends a phone photo of a skin lesion plus a short text history. Path to Care runs the image and narrative through a multimodal agentic pipeline and returns:

Top-3 conditions with confidence — never a single class label.
Red / Yellow / Green urgency, with the rule-based safety net only ever escalating, never relaxing.
Structured SOAP note for the clinic physician — chief complaint, HPI, exam, red flags, patient concerns.
Patient framing that names the cost-of-care tradeoff in the patient's own numbers (transport ₹, daily wage ₹, distance km).

Every patient-facing string passes through a deterministic cardinal-rule rewriter that strips diagnostic phrasing ("you have X" → "signs suggest X") before it leaves the API. The output is decision support, not a diagnosis.

Architecture

Multimodal Gemma 4 31B-it serves both the image classifier and the triage reasoner from one set of weights on a single AMD Instinct MI300X (192 GB VRAM, ROCm).
Qwen 2.5-7B-Instruct handles text-only SOAP extraction via a DSPy NarrativeToSOAP signature.
LoRA SFT on MI300X — supervised fine-tune of the triage head, 32 seconds wall-clock, loss 3.90 → 0.58 in one epoch. 180 MB adapter, served via vLLM --enable-lora.
DSPy ReAct orchestrator coordinates five MCP servers: image classifier, SOAP extractor, village context, triage reasoner, and camera capture.
vLLM (ROCm) in Docker for serving — OpenAI-compatible API on :8000. The Next.js frontend talks to it directly.
Adversarial test set — 30 hand-authored cases (10 R / 10 Y / 10 G) with red-flag, contradiction, and off-distribution variants. Held out from training.

Track 1 · Agents Track 2 · Fine-Tuning MI300X Track 3 · Multimodal Qwen prize Hugging Face prize Build-in-Public

Results

Evaluated on the 30-case adversarial test set. Reward R = 1.0 exact / 0.5 adjacent / 0.0 off-by-two.

Run	Mean reward	Exact match	FN Red → Green
Zero-shot baseline (Gemma 4 31B)	0.983	96.7%	0.0%
LoRA-tuned (180 MB adapter)	0.983	96.7%	0.0%

The single residual error is a Yellow → Green slip; no Red was missed. The headline number is the false-negative Red → Green rate at zero — the safety property that matters in the field. Full per-case results in results/tuned_metrics.json.

Try it

Live demo on Hugging Face

Public, always-on Space. Upload a photo or describe in text — the Space HTTPs into the MI300X-hosted vLLM container for all model work.

Source on GitHub

Full repo: orchestrator, MCP servers, LoRA training, eval harness, adversarial generator, Next.js frontend.

LoRA adapter on HF Hub

The 180 MB triage adapter trained on MI300X. Load with PEFT or serve via vLLM --enable-lora.

The cardinal rule

Path to Care never produces diagnostic statements. The output is always "signs suggest infection", never "you have cellulitis"; image output is always top-3 with confidence, never a single class label, never binary sick / healthy. Enforcement is defense-in-depth: a system prompt rule, a deterministic regex rewriter on every model output, and a unit test suite that fails the build on diagnostic phrasing.

Decision-support tool. Not a diagnostic system. Always consult a qualified physician.