# PsychScribe Local

A local-first clinical-scribe concept for a solo psychiatrist — capture sessions, produce clinician-reviewed notes, strictly offline by default.

Status: planned

## The Problem

Clinical documentation eats a private-practice psychiatrist's day, and the cloud "AI scribe" tools that promise relief route patient audio and notes through someone else's servers. For a solo clinician handling sensitive psychiatric sessions, strict local/offline operation isn't a nice-to-have — it's the precondition for using the tool at all.

## What I Built

PsychScribe Local is a macOS desktop concept for capturing clinical sessions and producing clinician-reviewed notes, with strict offline operation as the default. The shared architecture is a local-first clinical scribe for physicians and therapists; the first specialty pack is psychiatry, optimized for medication-management and therapy visits.

The repo is further along than a spec scaffold on the app-slice track: `PsychScribeCore` is a real Swift package whose test suite passes against real GRDB 6.29.3 and WhisperKit 0.18.0, and a SwiftUI macOS app builds and runs an Import → Transcribe → Review → Draft → Export slice end to end. Live chunked microphone capture is a compiling stub; file import is the working capture path for v0. Transcript clinical accuracy is explicitly unvalidated until a large-v3 benchmark run on target M1 hardware — the review gate described below is the safety boundary until then.

## Architecture

One imported audio file traces through seven stages, all local:

1. **Import** — `AudioImporter` reads the source file; live mic capture goes through `SessionRecorder`/`ChunkCrypto` but recording itself is a stub in v0, so import is the working entry point.
2. **Chunked transcription** — `ChunkedTranscriptionPipeline` reads the file in fixed 30-second windows with a 3-second trailing overlap (mono 16 kHz Float32) rather than decoding the whole file into RAM, because a single long `transcribeFile` call was measured at ~3.8 MB/min of decoded audio with no bound for multi-hour sessions. Each window is transcribed in isolation by `WhisperKitTranscriber`, so only one window plus one temp file exist in memory/disk at a time regardless of total session length.
3. **Stitching** — `TranscriptStitcher` reconciles the overlapping regions between adjacent windows by timestamp proximity and text similarity into one `TranscriptDocument` in a provisional state.
4. **Transcript review** — the clinician edits segment text and marks the transcript approved in the Transcript Review UI. `ReviewGateEngine.canExport` blocks everything downstream — note generation and export — until `transcript.state == .approved`.
5. **Note drafting** — once approved, `OllamaNoteGenerator.draftNote` sends the transcript and a `NoteTemplate` to a local Ollama server (`localhost:11434`, OpenAI-compatible `/v1/chat/completions`, falling back to `/api/chat`) running `llama3.2:3b`, with a system prompt that forbids stating anything not present in the transcript and requires `[clinician to confirm]` placeholders for anything the model can't source.
6. **Deterministic safety flagging** — `ClinicalFactFlagger`, a regex/lexicon scanner with no LLM involved, runs twice: once over the raw transcript, once over the LLM-drafted note text. It flags every medication name, dose/frequency pattern, diagnosis cue, risk/SI/HI/self-harm phrase, safety-plan mention, and level-of-care phrase as a pending `ReviewGate`. This is the actual export gate: `ReviewGateEngine` blocks Markdown/TXT export until every gate is resolved (a dev-only override exists, labeled as such in the UI) — the LLM drafts, the deterministic scanner is what enforces the AGENTS.md hard rule that clinical content requires human review before it leaves the app.
7. **Export** — `MarkdownExporter` (v0) or `TranscriptTextExporter` writes an `ExportArtifact` to local disk; no network call is made at this stage. PDF export is planned, not yet implemented.

Audio is encrypted at rest throughout via `ChunkCrypto` (AES-GCM/CryptoKit, symmetric key in the macOS Keychain, custom authenticated file format `PSENC002`), decrypted only transiently in memory for transcription. Structured records (`Project`, `Encounter`, `TranscriptDocument`, `ClinicalNote`) persist through `GRDBClinicalStore` — GRDB 6 over plain SQLite, one JSON-blob column per row keyed by id (encounter-indexed for transcripts/notes). SQLCipher (DB-at-rest encryption) is deferred to a named follow-up hardening step (ADR 0005); v0 accepts plain SQLite because only synthetic, non-PHI fixtures are used — the higher-sensitivity artifact, audio, is already encrypted independently.

A separate track, `tools/asr-bench`, benchmarks WhisperKit models against synthetic non-PHI fixtures (WER, medication-term recall, real-time factor) to decide which model is clinically viable for the accuracy pass — the first real benchmark run, `openai_whisper-tiny` over 6 fixtures, measured 52.8% avg WER and 48.1% avg term recall, missing most medication names and dose values, which is why the app default stays `tiny` for dev/demo and `large-v3` is the named target for a real accuracy pass, not yet run on target hardware.

```mermaid
flowchart LR
  A["Imported audio file<br/>(mic capture is a stub)"] --> B["ChunkCrypto<br/>AES-GCM, Keychain key"]
  B --> C["ChunkedTranscriptionPipeline<br/>30s windows, 3s overlap"]
  C --> D["WhisperKitTranscriber<br/>tiny default -> base/small/large-v3 toggle"]
  D --> E["TranscriptStitcher<br/>overlap reconciliation"]
  E --> F[("GRDBClinicalStore<br/>SQLite, JSON blobs")]
  E --> G["Transcript Review UI<br/>clinician edits + approves"]
  G --> H{"ReviewGateEngine<br/>blocks until approved"}
  H --> I["OllamaNoteGenerator<br/>llama3.2:3b @ localhost:11434"]
  I --> J["ClinicalFactFlagger<br/>deterministic regex/lexicon scan<br/>meds, doses, dx, SI/HI, safety-plan"]
  J --> K["Note Review UI<br/>resolve pending gates"]
  K --> L{"ReviewGateEngine<br/>blocks export until gates resolved"}
  L --> M["MarkdownExporter / TranscriptTextExporter<br/>local disk, no network"]
  N["tools/asr-bench<br/>WER / term-recall / RTF harness"] -.picks model for.-> D
```

**Models — which, where, why:**

| Stage | Model / tier | Where it runs | Why there |
|---|---|---|---|
| Transcription | WhisperKit; default `openai_whisper-tiny`, with a UI toggle to `base` / `small` / `large-v3` | On-device, `WhisperKitTranscriber` (Argmax WhisperKit 0.18.0) | Native Apple Silicon Whisper port; `tiny` is the dev/demo default and the only model benchmarked so far — the first real run missed most medication names and doses, confirming `large-v3` is required before any clinical use |
| Note drafting + fact extraction | Ollama `llama3.2:3b` | Local Ollama server, `localhost:11434`, OpenAI-compatible `/v1/chat/completions` (falls back to `/api/chat`) — `OllamaNoteGenerator` uses `URLSession` only, no SDK | Keeps note generation off any network; a 3B model fits the M1/16 GB target machine; the system prompt bans stating anything not present in the transcript |
| Clinical safety flagging | Models: none — deterministic regex/lexicon scan | `ClinicalFactFlagger`, in-process, run over both the transcript and the LLM-drafted note | Medication/dose/diagnosis/risk/SI-HI/safety-plan detection has to be reliable and auditable, not probabilistic — this is the actual export-blocking check, independent of whatever the LLM drafted |

**Tools & infra — which, why:**

- **Storage** — GRDB 6 over SQLite for structured records (`Project`/`Encounter`/`TranscriptDocument`/`ClinicalNote`, JSON-blob columns); plain SQLite in v0, not SQLCipher, because only synthetic non-PHI fixtures exist during prototyping — DB-at-rest encryption is a tracked next step (ADR 0005).
- **Audio encryption** — CryptoKit AES-GCM with the symmetric key held in the macOS Keychain (`Security` framework), a custom authenticated file format (`PSENC002`) versioned past an earlier `PSENC001`; audio is the highest-sensitivity artifact so it's encrypted independently of the DB-at-rest gap above.
- **Local LLM runtime** — Ollama, reached over plain HTTP at `localhost:11434`; no vendor SDK, just `URLSession` against the OpenAI-compatible chat endpoint.
- **ASR engine** — WhisperKit (Argmax) 0.18.0 as a Swift package dependency; downloads the selected model once, then runs fully offline.
- **App shell** — native SwiftUI macOS app, `XcodeGen`-generated Xcode project (`xcodegen generate` + `xcodebuild`) — chosen over the Tauri + React alternative weighed in the tech-stack direction doc, in favor of direct WhisperKit/Keychain/AVFoundation integration.
- **Benchmark harness** — `tools/asr-bench`, a standalone Swift executable scoring WhisperKit models against synthetic (non-PHI) fixtures via `WERScorer` / `TermRecallScorer` / `BenchMetrics` (WER, medication-term recall, real-time factor, peak RSS).
- **Hosting** — none; the app is 100% local with no cloud deploy. The repo's "Cloud Boundary" policy requires explicit opt-in, a signed BAA, and documented data flow before any cloud ASR/LLM/telemetry vendor can be added.

## How it works

1. The clinician imports a session audio file (live chunked mic recording exists as a stub, not the working path yet).
2. `ChunkedTranscriptionPipeline` reads the file in 30-second windows with a 3-second overlap, transcribing each window through `WhisperKitTranscriber` (default `openai_whisper-tiny`) so memory use stays bounded regardless of session length.
3. `TranscriptStitcher` reconciles the overlapping regions by timestamp and text similarity into one provisional `TranscriptDocument`.
4. The clinician reviews and edits the transcript in the Transcript Review UI and marks it approved; `ReviewGateEngine` blocks every step past this point until that approval is set.
5. `OllamaNoteGenerator` sends the approved transcript plus a psychiatry note template to a local Ollama server (`llama3.2:3b`) and gets back a structured `ClinicalNote` draft, with sections that cite transcript spans or are marked `[clinician to confirm]`.
6. `ClinicalFactFlagger` deterministically re-scans both the transcript and the drafted note for medication names, doses/frequencies, diagnosis cues, SI/HI/self-harm/safety-plan language, and level-of-care phrases, producing a pending `ReviewGate` for each hit.
7. The clinician resolves every pending gate in the Note Review UI; `ReviewGateEngine.canExport` only returns `allowed: true` once the transcript is approved, all gates are resolved, and consent wasn't declined (a labeled dev-only override exists to unblock prototype work).
8. `MarkdownExporter` or `TranscriptTextExporter` writes the approved note to local disk (Markdown/TXT in v0; PDF planned) — no network call happens at export.
9. Independently, `tools/asr-bench` runs WhisperKit models against synthetic fixtures to measure WER, medication-term recall, and real-time factor, informing which model is promoted to the default before any real clinical use is considered.

## Tech stack

SwiftUI macOS app (XcodeGen project) plus a shared `PsychScribeCore` Swift package consumed by both the app and the `asr-bench` harness. WhisperKit (Argmax, 0.18.0) for on-device transcription. Ollama (`llama3.2:3b`, local HTTP, OpenAI-compatible endpoint) for note drafting — no cloud LLM. GRDB 6 over SQLite for structured storage; CryptoKit AES-GCM plus macOS Keychain for audio-at-rest encryption. `ClinicalFactFlagger` is a from-scratch deterministic regex/lexicon scanner, not a library or a model. No cloud services, no hosting — the app runs entirely on-device.

## Locked Defaults

- **Platform** — macOS desktop first (iPhone/iPad later), sized for an M1 MacBook Pro running back-to-back visits across a full clinical day.
- **PHI posture** — strict local/offline by default; audio retained encrypted only until the transcript and note are approved, then deleted.
- **Clinician in the loop** — notes are drafted for review, never auto-filed.

## Status and Boundaries

The repo pairs a product/specification scaffold (design docs, ADRs, compliance planning) with a compiling, tested app-slice prototype (see Architecture above). Neither makes it clinical software: it is **not** HIPAA certified, transcript accuracy is unvalidated on target hardware, and it must not be used with real patient data until the compliance-evidence package and clinical-validation gates are complete. The portfolio entry documents the product direction and the privacy-first architecture, not a shipping medical device.

## Links

- [HTML project page](https://rosslabs.ai/projects/psychscribe/)