Aida

Problem

A solo practitioner spending 12 hours a week on scheduling and billing has no cheap way to decide what to do about it. Hiring a consultant to run a multi-criteria decision analysis is disproportionate to the question. Asking a general chatbot produces a confident-sounding answer with no visible reasoning, no way to check the trade-off it made, and no guardrail against it drifting into clinical judgment or handling protected health information it was never supposed to see. The practitioner is left choosing between an unaffordable formal process and an unauditable informal one.

Concretely: “should I hire a part-time virtual assistant, outsource billing, hire onsite, upgrade tooling, or wait 60 days?” is a five-option, four-criteria decision (hours recovered, cost, quality, reversibility) that most people either resolve on gut feel or don’t resolve at all — the pain just continues.

Approach

Aida’s bet is that the LLM should never be trusted to produce the ranking itself. It classifies and proposes; deterministic TypeScript computes every score, weight, and rank. Concretely: Groq calls emit categorical tags and short rationale text only (values, weight-adjustment factors, feasibility tiers); every number that determines the recommendation — normalization, weighted sums, concordance, closeness coefficients — is a pure function with no model in the loop, and is unit-tested against known-correct references (Saaty’s textbook AHP example, hand-checked TOPSIS cases).

What was deliberately not done: no continuous-optimization solver (Aida declines “what’s the optimal price?” and reframes it as a discrete choice), no sequential/multi-step planning engine in v1 (a weekly-audit mode is scoped out), and no attempt to have the LLM “show its work” by explaining a score after the fact — the trace shown to the user is the actual computation, not a post-hoc rationalization of an opaque model output. PHI is rejected at the intake schema rather than filtered downstream, because a downstream filter that misses one case is a HIPAA incident and an upstream schema rejection that over-blocks is just a bad UX day.

Architecture

flowchart LR
    A[POST /api/decisions] --> B{Session actor?}
    B -->|no| C[Guest bucket]
    B -->|yes| D[Better Auth actor]
    C --> E[Rate limit\n20/24h]
    D --> E
    E --> F[Zod DecisionInputSchema\nrejects PHI-shaped free text]
    F --> G[runDecision orchestrator]
    G --> H[Stage 0-8 pipeline\nGroq: openai/gpt-oss-120b]
    H --> I{Guest?}
    I -->|yes| J[Return output\nno persistence]
    I -->|no| K[Neon Postgres via Drizzle\nRLS-scoped transaction]
    K --> L[decisions row +\naudit event per Groq call]

The API route runs the engine before opening a database transaction, deliberately: two of the nine stages call Groq (~1.5s round-trip each), and holding a Postgres transaction open across two LLM calls would tie up a connection for no reason. The engine itself makes no database writes, so it’s safe to run outside the actor-scoped transaction; only the persistence step (decision row + one audit-log row per Groq call, tagged with stage, token counts, and model) runs inside it. The API route also forces nodejs runtime rather than Edge, because Edge breaks the Neon WebSocket pool that row-level security depends on. Guests get the full engine output but nothing is written to Postgres — the client falls back to session-storage for a preview page.

How it works

Aida has two paths sharing the same deterministic core: the automation-finder path (pain point → ranked scaffold) and the decision path (named options → ranked recommendation). The decision path is the full pipeline:

Stage 0 — runStage0Classifier (PEDE classifier). Groq, temperature: 0, tags the question on two axes: epistemicType (descriptive / diagnostic / predictive / decision_analysis / optimization / sequential) and structuralType (finite option list, values-dominant, generative, exploratory, time-critical). Anything that isn’t a discrete decision_analysis gets declined and reframed rather than forced through the engine — asking “why did my no-show rate go up?” gets redirected to a forward decision, not answered.
Stage 1 — runStage1Values (Value-Focused Thinking). Groq reads the intake and emits a weightAdjustments factor per criterion in [0.5, 2.0] — never a new criterion, never a raw weight. TypeScript multiplies each template’s defaultWeight by its factor and re-normalizes to sum to 1. This is the only place the model touches the numbers, and it can only rebalance what already exists.
Stage 1b — runStage1bAhp (optional, Analytic Hierarchy Process). When the user does pairwise comparisons on Saaty’s 1–9 scale instead, weights come from the matrix’s principal eigenvector via power iteration (falls back to row-geometric-mean if it doesn’t converge in 50 steps). Consistency Ratio CR = CI / RI[n] where CI = (λmax − n) / (n − 1); CR > 0.10 flags the most-inconsistent pair for the user to revise rather than silently accepting contradictory input.
Stage 2 — runStage2Constraints (veto filtering, deterministic, no LLM). Each template constraint is a hard rule — e.g. the admin-hire template vetoes an onsite hire when monthsSavingsRunway < 3. Vetoed candidates are dropped with a stated reason before any scoring happens.
Stage 3 — runStage3Weights. Thin normalization pass — fills any criterion Stage 1 dropped from its default, re-sums to 1.
Stage 4 — runStage4Outranking (ELECTRE-style pairwise dominance). For every candidate pair, computes a weighted concordance score (fraction of weight where A ≥ B) and a discordance score (largest single-criterion gap where B beats A). A dominates B — and B is eliminated — when concordance ≥ 0.7 AND discordance ≤ 0.2.
Stage 5 — runStage5Ranking (TOPSIS). Each criterion column is divided by its Euclidean norm, multiplied by its weight, and compared against the ideal-best and ideal-worst vectors across the surviving candidates. Closeness coefficient = d⁻ / (d⁺ + d⁻); highest closeness wins. Confidence is round(50 + 50 × margin) where margin is the top-1/top-2 closeness gap, floored at 25 (a near-tie is flagged, not hidden). A separate minimax-regret pass picks the “robust alternative” — the option with the smallest worst-case weighted regret across criteria — for the “if this stops working” fallback.
Stage 6 — runStage6Feasibility. Groq classifies each proposed workload-reducer into one of four tiers (skill / plugin / agent / human) with temperature: 0; TypeScript then computes feasibilityScore from a fixed lookup (skill=100, plugin=80, agent=55, human=0) and impactScore from min(7, hrs/week) → 0–70 plus a coverage bonus (full_task=30, partial_task=18, task_setup=8), giving combinedScore = round(impactScore × feasibilityScore / 100).
Stage 7 — runStage7Scaffold. Deterministic, no LLM. Fires only for reducers tiered skill or plugin, generating the paste-ready SKILL.md / prompt / plugin layout.
Stage 8 — runStage8Promotion. Maps the recommendation onto an adoption-pathway ladder (prompt → checklist → skill → plugin → agent), capping at 2 “recommended” rungs to avoid decision overload and downgrading complex rungs when confidence is under 50.

The automation-finder path skips the MCDA pipeline and instead runs scoreCandidates() — a 9-criterion deterministic scorer (pain severity, frequency, time burden, business impact, AI fit, risk, data readiness, adoption friction, setup effort), each criterion scored 0–1 and combined via a weighted sum (default weight 1/9 each, min-direction criteria like risk and friction inverted before weighting).

Tech stack

Next.js 16 (forced nodejs runtime on the decisions route), TypeScript, Zod (schema validation and PHI rejection at intake), Groq running openai/gpt-oss-120b for the categorical/classification calls only, Neon Postgres + Drizzle with actor-scoped row-level security, Better Auth for session identity. No math library — AHP and TOPSIS are hand-implemented per a build-from-scratch constraint.

Results

Worked example, computed from the live admin-hire template’s real candidate scores and default weights (timeRecovered 0.35, cost 0.25, quality 0.2, risk 0.2), reproducing the exact Stage 4 / Stage 5 functions:

Candidate	timeRecovered	cost	quality	risk	TOPSIS closeness
Part-time virtual assistant	0.75	0.65	0.65	0.70	0.763 (winner)
Outsource billing	0.60	0.55	0.85	0.80	0.657
DIY better tooling	0.40	0.85	0.60	0.85	0.446
Defer 60 days	0.15	0.95	0.50	0.90	0.252

(Onsite hire dropped at Stage 2 — vetoed by the monthsSavingsRunway < 3 constraint in this scenario.) Stage 4 ELECTRE found no pairwise dominance strong enough to eliminate a candidate outright at this input, so all four survived to TOPSIS. Margin between rank 1 and rank 2 (0.763 − 0.657 = 0.106) yields confidence = 55.

✅ Verified: the AHP module (tests/ahp.test.ts, 10 tests) reproduces Saaty’s textbook 4-criteria eigenvector (Price/Size/Location/View) within the test’s tolerance, confirmed by running the suite (vitest run tests/ahp.test.ts → 10/10 passed). The table above was computed by running the actual pairScore and computeTopsis logic against the template’s checked-in candidate data, not hand-calculated.

⚠️ No production benchmark yet on the metric that matters — practitioner time saved after adopting a recommendation. The engine’s internal math is verified; real-world adoption and time-back are unmeasured.

Lessons

Categorical LLM output, numeric TypeScript output was chosen over letting the model emit scores directly, because a model asked to output a 0–1 score cannot be unit-tested against a known-correct answer the way CI = (λmax − n)/(n − 1) can — determinism and auditability required moving every number out of the prompt and into a pure function.
Running the engine before opening the database transaction was chosen over wrapping engine-plus-persist in one transaction, because two sequential Groq round-trips (~1.5s each) held inside an open Postgres transaction would burn connection time for no benefit — the engine makes no DB writes, so it doesn’t need the transaction’s isolation guarantees.
Declining out-of-scope question types (diagnostic, predictive, optimization, sequential) at Stage 0 was chosen over trying to answer them with the same MCDA machinery, because forcing a “why did X happen?” question through a ranking pipeline built for discrete choices would produce a plausible-looking but structurally wrong answer — reframing to a decision the pipeline can actually solve is more honest than a confident non-answer.