Digital Twins for Mental Health Clinician Training
AI-powered patient simulations using large language models enhanced with Conceptors — advanced steering vectors enabling precise control over psychiatric symptom presentations — for scalable, equitable clinician training worldwide.
The Problem & Our Approach
Mental health training faces a fundamental bottleneck: trainees need diverse clinical exposure, but access to patients with rare symptom combinations and cultural variations is limited by ethics, logistics, and cost. STELLAR addresses this through controllable digital patient twins.
Conceptor-Steered Digital Twins
We use Conceptors — a mathematical framework for manipulating internal LLM representations — to create patient simulations with precisely calibrated psychiatric symptoms across anxiety, depression, psychosis, and fear spectra. Unlike prompting, Conceptors enable continuous intensity control and composability via Boolean operations.
Philadelphia Neurodevelopmental Cohort
STELLAR leverages the PNC, a community sample of ~10,000 genotyped youth with structured clinical interviews, neurocognitive assessments, and neuroimaging. The recorded clinical interviews provide both ground-truth diagnostic labels and natural language exemplars of symptom expression.
Diagnostic Tool Robustness
Controlled digital twins serve as reproducible testbeds for evaluating emerging language-based diagnostic tools before clinical deployment — measuring demographic invariance and identifying bias across sex/gender, age, ethnicity, SES, and language proficiency.
Shareability & Global Reach
Once validated, symptom-specific Conceptors can be distributed to other institutions without retraining, accelerating adoption worldwide. All outputs — Conceptor libraries, code, and evaluation frameworks — will be released under open-source licenses.
Investigators & Collaborators
A collaboration uniting machine learning, psychiatry, speech science, and lived experience across NYU, Penn, and the Linguistic Data Consortium.
Lead PI
João Sedoc
Neville Ryant
Local PI — Penn Medicine
Raquel Gur
Local PI — Penn Engineering
Sharath Chandra Guntuku
Co-PI
Monica Calkins
Co-PI
Tyler Moore
Sunghye Cho
Rachel Gordon
Mark Liberman
Collaborator
Ruben Gur
Collaborator
Dominic A. Sisti
Foundational Work
Key publications by team members that underpin STELLAR's approach.
Unique Signatures in Verbal Fluency Task Performance in Schizophrenia and Depression
The Philadelphia Neurodevelopmental Cohort: Perspective, Lessons, and Future Directions
Large Language Models Could Change the Future of Behavioral Healthcare: A Proposal for Responsible Development and Evaluation
Key Language Markers of Depression on Social Media Depend on Race
Speech Markers of Depression Dimensions Across Cognitive Status
An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives
Conceptor-Aided Debiasing of Large Language Models
The Third DIHARD Diarization Challenge
Natural Language Processing Methods Are Sensitive to Sub-Clinical Linguistic Differences in Schizophrenia Spectrum Disorders
Burden of Environmental Adversity Associated With Psychopathology, Maturation, and Brain Behavior Parameters in Youths
The Second DIHARD Diarization Challenge: Dataset, Task, and Baselines
The Philadelphia Neurodevelopmental Cohort: Constructing a Deep Phenotyping Collaborative
Open Positions
We're hiring across both sites. STELLAR is a two-year project spanning NLP, speech processing, clinical psychiatry, and human-centered AI. We're looking for people who are excited about building tools that make mental health training more equitable and scalable.
Postdoctoral Researcher — NLP & Steering Vectors
Develop and validate Conceptor-based steering methods for LLMs. Strong background in representation learning, mechanistic interpretability, or activation engineering. Experience with clinical NLP a plus.
Research Scientist — Speech & Multimodal AI
Work on acoustic biomarker extraction, speaker diarization, and multimodal Conceptor integration from clinical interview recordings. Experience with wav2vec, Whisper, or clinical speech analysis preferred.
Research Engineer — Platform Development
Build the digital twin training platform — integrating Conceptor-steered LLMs with avatar rendering (SoulMachines), session recording, and automated feedback systems. Full-stack experience with ML deployment.
Research Coordinator — Clinical Evaluation
Coordinate stakeholder surveys, focus groups, and the trainee-tool-trainer implementation readiness study. Experience with IRB protocols, mixed-methods research, and mental health populations.
Interested? Fill out this brief interest form with your CV and a note on what draws you to this work. We'll be in touch.