Data Intel
Local-first research pipeline that plans, checks sources, connects evidence, and produces cited analysis through a CLI or local dashboard.
The problem
Complex research jobs need more than a single answer. Sources must be collected within an approved scope, claims must retain their provenance, and the final report must make gaps and contradictions visible. A long agent conversation alone does not provide a durable record of those steps.
What I built
Data Intel is a local-first Python pipeline with six stages: plan, collect, assess, link, insight, and write. A coordinator records each run in inspectable files and an append-only event log. The CLI drives the stages, while a dashboard bound to 127.0.0.1 lets one user create a run, review inputs and plans, follow progress, and inspect outputs. The dashboard displays commands for stage execution; it does not run stages in the browser.
The model router can use a local endpoint or an approved provider. Its local-only setting defaults to on and fails closed if configuration cannot be read. Under host mode, Claude Code or Codex answers a schema-bound stage packet instead of the Python process making a model call. There is no MCP server.
flowchart LR
A[Research request] --> B[Plan and approval]
B --> C[Collect sources]
C --> D[Assess claims]
D --> E[Link entities and evidence]
E --> F[Find gaps and trends]
F --> G[Write cited outputs]
G --> H[Check acceptance criteria]Evidence and limits
Each claim atom records a source and character span. The write stage produces files even if citation or rubric checks fail, so the coordinator checks the stage verdict before marking a run complete. The dashboard is local and has no authentication; exposing it outside loopback requires an explicit override.
The repository records Phases 1–4 as implemented. Phase 5’s three proving research jobs remain the acceptance step for end-to-end quality. A working pipeline is not yet evidence that its generated research is reliable across those jobs.