Voice-Based Patient Character Simulator

A low-latency, multi-agent voice-based patient simulator that lets medical students and residents practice diagnostic conversations and get treatment feedback in real time. The system pairs a React front end with a FastAPI backend, containerized with Docker for consistent local and cloud deployment.

To keep the simulated patient's answers grounded rather than generic, I built an AI-powered context-retrieval pipeline using LlamaIndex and ChromaDB, so the underlying LLM stays anchored to a structured knowledge base of patient histories and symptoms instead of improvising. Tuning the retrieval and prompt layer brought end-to-end response time under 200 ms and shortened practitioner training time by roughly 10%.

The live demo is currently offline (the free-tier hosting it ran on has lapsed). The screenshots below walk through the conversation flow, from a doctor starting a session to the simulator relaying patient responses in real time.