ML Speech Analysis & Dashboard Upgrade

Context & What’s Already in Place
Mobile app (React Native): Patients record voice notes → uploaded to storage/audio/<entryId>.m4a → AssemblyAI transcribes to rawText → GPT “rewrites for dementia readability” → Firestore doc entries/{entryId} with:

rawText (ASR output)
text (GPT result)
audioURL, imageURL
prettyDate & timestamp

Clinician Dashboard (Web UI): Lists “patients” → shows charts & tables of entries, but needs richer speech metrics and better navigation to aid clinicians.

I need speech analysis for the patient recorded entires:

Words-Per-Minute (WPM)
What: Total words spoken ÷ total time (including pauses).
Why: A sustained drop in WPM can reflect slowed cognitive processing and word retrieval difficulties.

Mean Pause Duration
What: Average length of silent gaps (e.g. >250 ms) between utterances.
Why: Longer pauses often indicate increased word-finding trouble or hesitancy—common in mild cognitive impairment.

Pause Density (Pauses Per Minute)
What: Number of pauses ÷ total recording time.
Why: More frequent pausing suggests difficulty planning or retrieving language.

Articulation Rate
What: Number of syllables spoken ÷ speaking time (excluding pauses).
Why: Captures how rapidly speech motor control operates; a declining rate can signal motor planning issues.

Lexical Diversity (Type-Token Ratio)
What: Unique words ÷ total words.
Why: People with early dementia tend to repeat the same words and use a more limited vocabulary.

Syntactic Complexity
What: Average sentence length or number of subordinate clauses per sentence.
Why: Reduced sentence complexity (short, simple sentences) can reflect declining working memory and planning.

Semantic Error Rate
What: Proportion of “circumlocutions” or incorrect word substitutions (e.g. saying “thingy” instead of item)
Why: Increased semantic errors are a hallmark of word-finding difficulty in dementia.

Prosodic Variation (Pitch & Intensity Range)
What: Range or standard deviation of fundamental frequency (pitch) and amplitude (loudness).
Why: Flatter, monotone speech and reduced loudness variation can accompany cognitive and motor changes.

For example JSON after recording an entry:
{
"audioURL": "https://firebasestorage.googleapis.com/…/audio/1750388268184.m4a",
"imageURL": null,
"rawText": "Um… I went to the park and I saw… squirrels and birds… and then I had tea.",
"text": "I went to the park, saw squirrels and birds, then had tea.",
"prettyDate": "Thursday, 19 June 2025",
"timestamp": { "seconds": 1_750_388_268 },
"stats": {
"wpm": 60,
"totalPauseTime": 8.4,
"meanPauseDuration": 0.70,
"pauseDensity": 5.6,
"articulationRate": 3.2,
"lexicalDiversity": 0.78,
"syntacticComplexity": 1.2,
"semanticErrorRate": 0.05,
"prosodicVariation": 35.4,
"riskScore": 0.42
}
...

The riskScore should be the probability output of a simple binary classifier (e.g. logistic regression) that’s been trained to distinguish “healthy” vs. “early-dementia” speech, using your extracted metrics as features. We will need to find dementia datasets that can be helpful.