Obsidian Vault LLM Wiki

The Problem

Most personal knowledge bases are readable by humans but hard for agents to use safely. The files contain useful context, but retrieval is inconsistent, raw source material mixes with synthesis, and agents can overreach when they are handed the whole vault.

The LLM Wiki separates durable knowledge from raw material, gives agents a narrow query path, and keeps private source data out of public surfaces.

What I Built

The Obsidian Vault LLM Wiki is a private, local-first knowledge system for turning notes, source material, decisions, projects, and operating context into a readable wiki that humans can browse and LLM agents can query.

It is not presented as a public repo because the value is the architecture and retrieval method, not the private note contents.

File Structure

The vault uses a small number of stable zones.

Obsidian Vault
|-- _system/      # workspace instructions, memory, indexes, logs, eval notes
|-- brain/        # operating rules and voice guidance
|-- wiki/         # compiled knowledge: concepts, entities, sources, work
|-- raw/          # immutable source material, filed by domain
|-- tools/        # scripts, commands, plugins, prompts, skills, agents
|-- templates/    # reusable note and entity templates
|-- outputs/      # drafts, health reports, QA, snapshots
|-- writing/      # author-owned writing and suggestion sidecars
`-- .vector/      # gitignored retrieval indexes and embeddings

The public description only names the system shape. It does not publish notes, source files, filenames from private material, or graph relationships.

Retrieval System

The retrieval layer is local and derived from the Markdown vault. The source of truth stays in Obsidian files with YAML frontmatter; the searchable stores are rebuilt from those files.

Vector store. The current implementation stores chunk embeddings in a gitignored .vector/embeddings.json file. Chunks are created from wiki pages and selected tool/control docs, then embedded locally by default with Ollama’s nomic-embed-text. The maintainer script can refresh only changed chunks and write a tracker that shows stale, changed, and pending material.

SQLite index. A sidecar .vector/wiki.db stores page metadata, chunks, aliases, graph edges, and SQLite FTS5 rows. FTS5/BM25 adds exact and keyword strength that dense vectors miss, especially for named entities, page IDs, and title-like queries.

Graph rerank. Authored frontmatter relationships and wikilinks are parsed into a graph. Search can use Personalized PageRank to boost pages connected to the best vector hits, while preserving vector and lexical relevance as the main signals.

Quality boundaries. Draft or low-confidence pages remain searchable but are downweighted at rerank. Personal-only and investment-scoped material is excluded from general professional queries unless the query explicitly asks for that scope. This is a retrieval-quality boundary, not a cryptographic privacy boundary.

Plugin Support

The Obsidian Vault Search plugin makes the retrieval system usable inside Obsidian.

It has three modes:

Mode	What It Does
Quick	Finds files and aliases quickly from the local vault index.
Facet	Filters frontmatter with queries like type:entity, status:sprout, or capability:embedding.
Semantic	Uses ?query to search the vector layer.

On desktop, semantic mode can shell out to vault_vector.py and use the full hybrid stack: local embeddings, FTS5/BM25, and graph rerank. On Mac or iOS, the plugin can run on-device with transformers.js and a local ONNX nomic-embed-text-v1.5 model, then compare the query embedding against .vector/embeddings.json.

That gives the vault two working surfaces: a terminal command for agents and a native Obsidian search flow for day-to-day use.

Agent Access Model

Worker LLMs get a read-only query surface rather than broad vault access. The stable command is tools/scripts/llmwiki, which can answer from the derived indexes, show raw retrieval hits, and inspect graph relationships.

Maintenance commands such as ingest, edit, embed, promote, supersede, and saved-output writes stay reserved for trusted maintainer sessions. That keeps the public story simple: agents can use the knowledge system without being handed the underlying private archive.

Impact

The system treats Markdown as the durable human-readable record and retrieval as a rebuildable layer. That keeps the vault portable while still giving agents structured access to context, evidence, and relationships.

The important idea is not “a vector database over notes.” It is a controlled knowledge workflow: raw sources become wiki pages, wiki pages become indexed chunks, retrieval combines vector, lexical, and graph signals, and agents only receive the narrow access path needed for the task.