alaya

ālaya (ālayavijñāna) — storehouse consciousness. The layer of mind that holds all impressions, memories, and seeds of knowledge.

A FastMCP server that makes Claude Code the primary interface for a personal knowledge vault (zk or Obsidian). Full read/write/search/synthesis access — the AI is the interface, the vault is the source of truth.

Quickstart

# 1. Register with Claude Code (one command, no clone needed)
claude mcp add alaya -e ALAYA_VAULT_DIR=$HOME/notes -- uvx alaya-mcp

# 2. Start Claude Code — alaya connects automatically
claude

That's it. Ask Claude to "search my notes for X" or "create a note about Y" and it works.

Vault setup

zk vault (requires zk CLI):

brew install zk
mkdir -p ~/notes && cd ~/notes && zk init

Obsidian vault — just point ALAYA_VAULT_DIR at any directory with a .obsidian/ folder. No extra tools needed.

Running from source

git clone [email protected]:luke-kucing/alaya.git
cd alaya && uv sync

# Register with Claude Code from source
claude mcp add alaya \
  -e ALAYA_VAULT_DIR=$HOME/notes \
  -- uv run --directory $(pwd) alaya

Running the server standalone

# Via make
make serve

# Or directly with env var
ALAYA_VAULT_DIR=~/notes uv run alaya

The server communicates over stdio (MCP protocol) — it's designed to be launched by Claude Code, not run in a browser.

Running tests

make test                # unit tests (607+ tests, no external deps)
make test-integration    # integration tests (requires zk binary)
make lint                # ruff check

Philosophy

• The AI is the interface. Stay in one place — a Claude Code session — and converse. Claude reads, writes, searches, and reasons across the vault.
• Frictionless capture, deliberate structure. Inbox capture is a single sentence. Processing, linking, and filing are separate structured acts.
• Propose, then act. Destructive or structural changes follow a confirmation flow. Appends and captures are always safe.
• Pluggable vault backend. ZkBackend delegates to the zk CLI; ObsidianBackend is pure Python. The vault stays portable either way.
• Semantic retrieval over keyword search. LanceDB hybrid search finds notes by meaning, not just exact words.

Architecture

┌──────────────────────────────────────────────────────┐
│                   Claude Code                        │
│              (MCP client / AI layer)                 │
│     Synthesis, weekly review, gap analysis,          │
│     confirmation flow — all Claude's responsibility  │
└───────────────────────┬──────────────────────────────┘
                        │ MCP protocol (stdio)
┌───────────────────────▼──────────────────────────────┐
│                  alaya (FastMCP server)               │
│                                                       │
│  ┌─────────────┐ ┌─────────────┐ ┌────────────────┐  │
│  │ tools/      │ │ tools/      │ │ tools/         │  │
│  │  read.py    │ │  write.py   │ │  search.py     │  │
│  │  inbox.py   │ │  edit.py    │ │  structure.py  │  │
│  │  tasks.py   │ │  ingest.py  │ │  external.py   │  │
│  │  capture.py │ │  stats.py   │ │  graph.py      │  │
│  └──────┬──────┘ └──────┬──────┘ └───────┬────────┘  │
│         │               │                │            │
│  ┌──────▼───────────────▼────────────────▼─────────┐  │
│  │              backend/ (VaultBackend protocol)    │  │
│  │  ┌──────────────────┐  ┌──────────────────────┐  │  │
│  │  │ ZkBackend        │  │ ObsidianBackend      │  │  │
│  │  │ (delegates to zk │  │ (pure Python,        │  │  │
│  │  │  CLI subprocess) │  │  no external deps)   │  │  │
│  │  └──────────────────┘  └──────────────────────┘  │  │
│  └──────────────────┬──────────────────────────────┘  │
│                     │                                  │
│  ┌──────────────────▼──────────────────────────────┐  │
│  │  Shared layer                                    │  │
│  │  vault.py (path safety)    zk.py (CLI wrapper)  │  │
│  │  config.py (env vars)      watcher.py (watchdog)│  │
│  │  events.py (pub/sub)       audit.py (JSONL log) │  │
│  │  cache.py  (metadata cache, O(1) lookups)       │  │
│  └──────────────────┬──────────────────────────────┘  │
│                     │                                  │
│  ┌──────────────────▼──────────────────────────────┐  │
│  │              index/          providers/          │  │
│  │  embedder.py (nomic ONNX+LRU)gitlab.py (glab)   │  │
│  │  store.py    (LanceDB)       github.py (gh)     │  │
│  │  router.py   (query class.)                     │  │
│  │  corrective.py (retry RAG)                      │  │
│  │  graph_rag.py  (link expand)                    │  │
│  │  hyde.py     (HyDE embed)                       │  │
│  │  contextual.py (chunk ctx)                      │  │
│  │  reindex.py  (rebuild)                          │  │
│  └──────────────────────────────────────────────────┘  │
└───────────────────────────────────────────────────────┘
         │                              │
    ┌────▼─────┐    ┌──────────────────▼──────────────┐
    │  zk CLI  │    │  External providers              │
    │ (notes,  │    │  glab CLI · gh CLI · Outline API │
    │  links,  │    │  (pull issues in, push ideas out)│
    │  tags)   │    └──────────────────────────────────┘
    └────┬─────┘
         │
    ┌────▼──────────────────────────────┐
    │  ~/notes (vault)                  │
    │  ├── daily/                       │
    │  ├── inbox.md                     │
    │  ├── projects/                    │
    │  ├── areas/                       │
    │  ├── people/                      │
    │  ├── ideas/                       │
    │  ├── learning/                    │
    │  ├── resources/                   │
    │  ├── raw/                         │
    │  ├── archives/                    │
    │  └── .zk/ or .obsidian/           │
    │      ├── vectors/ (lance)         │
    │      └── audit.jsonl              │
    └───────────────────────────────────┘

Search Pipeline

Alaya's search is a multi-stage RAG pipeline, not a simple keyword lookup:

Query ──▶ Router ──▶ Retrieval ──▶ Correction ──▶ Expansion ──▶ Results
          │          │              │               │
          │          │              │               └─ Graph RAG (opt-in)
          │          │              │                  wikilink traversal
          │          │              │
          │          │              └─ Corrective RAG
          │          │                 score check → reformulate → retry
          │          │
          │          └─ Hybrid Search (vector + BM25 + RRF)
          │             or Keyword-only / Semantic-only
          │
          └─ Adaptive Router
             KEYWORD | SEMANTIC | TEMPORAL | HYBRID

Search features

Design decisions

Plain functions, not classes. Tools are async def get_note(path, vault) -> str — no inheritance, no base classes. State is passed explicitly. A Chunk dataclass is the only data container.

VaultBackend as the seam. Tools call backend.list_notes(), backend.search(), etc. ZkBackend delegates to the zk CLI via run_zk(); ObsidianBackend is pure Python. Unit tests mock alaya.zk.run_zk for zk paths, or use real filesystem fixtures for Obsidian paths. Wiring tests prove each tool dispatches through the backend.

LanceDB is additive. Core tools work without any index. Search falls back to backend keyword search. The vector index is an optimization layer, not a requirement.

Tools return Markdown strings. Claude reads Markdown; it doesn't parse JSON. Tables, bullet lists, and metadata headers are formatted for Claude consumption.

Confirmation lives in Claude, not the server. The server executes; Claude proposes and waits for user approval on destructive ops.

External systems are bridges, not mirrors. The vault doesn't replicate GitLab/GitHub/Outline — it pulls context in and pushes ideas out.

Tools

Read & navigate

Write & capture

Search

Edit & structure

Ingest & external

External issue CRUD (list, close, update) is handled by Claude Code natively via glab/gh CLI — the MCP server only bridges the vault to external systems.

Index Pipeline

Chunking strategies

Embedding

• Model: nomic-embed-text-v1.5 via fastembed (ONNX, no PyTorch required)
• Dimensions: 768
• Prefixes: search_query: for queries, search_document: for chunks
• Quantized variant: nomic-v1.5-q4 available via ALAYA_EMBEDDING_MODEL env var
• Hot-swap: Changing the model triggers automatic background re-embedding

Write-through indexing

All write operations (create, append, edit, move, rename, delete) emit events that trigger immediate index updates. The file watcher catches external edits with 2-second debounce. Double-indexing is prevented via coordination between the event system and watcher. A VaultMetadataCache keeps frontmatter, tags, and wikilinks in memory for O(1) lookups — automatically invalidated by both the event system and file watcher.

Audit logging

Every tool call is logged to audit.jsonl in the vault data directory (.zk/ or .obsidian/) with timestamp, tool name, arguments, status, duration, and result summary. The TUI dashboard tails this file for real-time activity monitoring.

Vault structure

~/notes/
├── daily/       # daily notes — connective tissue
├── inbox.md     # frictionless capture, process weekly
├── projects/    # active work with a finish line
├── areas/       # ongoing responsibilities
├── people/      # 1:1s, context, relationships
├── ideas/       # not ready to be a project yet
├── learning/    # active study
├── resources/   # settled reference, distilled knowledge
├── raw/         # drop zone for PDFs and URLs (auto-ingested)
└── archives/    # completed or dead things (soft-delete target)

Directory names are configurable via alaya.toml (see Configuration below). Notes use minimal YAML frontmatter (title + date) and inline #tags. Links are wikilinks ([[title]]).

Stack

Configuration

Prerequisites

• Python 3.12+
• uv — brew install uv
• zk — brew install zk (optional, only needed for zk vaults)
• glab — brew install glab (optional, for GitLab bridge)
• gh — brew install gh (optional, for GitHub bridge)

Environment variables

alaya.toml (optional)

Place an alaya.toml in your vault root to override defaults:

[vault]
type = "obsidian"  # or "zk" — overrides auto-detection

[directories]
person = "people"
idea = "ideas"
project = "projects"
learning = "learning"
resource = "resources"
daily = "daily"

[settings]
archives_dir = "archives"
default_capture_dir = "inbox"
default_external_dir = "external"

All fields are optional. Without this file, alaya auto-detects the backend from .zk/ or .obsidian/ and uses sensible defaults.

Optional dependencies

# Install cross-encoder reranking support
uv sync --extra rerank

Pass additional env vars when registering with Claude Code using -e:

claude mcp add alaya \
  -e ALAYA_VAULT_DIR=$HOME/notes \
  -e GITHUB_REPO=owner/repo \
  -- uvx alaya-mcp

Configure any combination of providers. pull_external and push_external auto-detect the provider from URLs or use the configured defaults.

Development

make install          # install dependencies
make test             # run unit tests (607+ tests)
make test-unit        # run unit tests verbose
make test-integration # run integration tests (requires zk binary)
make lint             # ruff check
make serve            # start the server

Project structure

src/alaya/
├── server.py           # FastMCP server, tool registration, audit wrapping
├── config.py           # env var loading, vault validation
├── vault.py            # path safety, frontmatter parsing
├── cache.py            # VaultMetadataCache: in-memory metadata index, O(1) lookups
├── zk.py               # zk CLI subprocess wrapper
├── events.py           # pub/sub event bus for write-through indexing
├── watcher.py          # watchdog file system monitor + cache invalidation
├── errors.py           # structured error codes
├── audit.py            # JSONL tool call logging
├── backend/
│   ├── protocol.py     # VaultBackend protocol + VaultConfig
│   ├── config.py       # vault type detection, alaya.toml loading
│   ├── zk.py           # ZkBackend (delegates to zk CLI)
│   └── obsidian.py     # ObsidianBackend (pure Python)
├── tools/
│   ├── read.py         # get_note, list_notes, backlinks, links, tags, reindex
│   ├── write.py        # create_note, append_to_note, update_tags
│   ├── inbox.py        # capture, get_inbox, clear_inbox_item
│   ├── capture.py      # smart_capture (auto-routing)
│   ├── search.py       # adaptive search with corrective RAG pipeline
│   ├── structure.py    # move, rename, delete, find_references
│   ├── edit.py         # replace_section, extract_section
│   ├── tasks.py        # get_todos, complete_todo
│   ├── external.py     # pull_external, push_external
│   ├── ingest.py       # URL/PDF/markdown ingestion
│   ├── stats.py        # vault_stats
│   ├── graph.py        # vault_graph (wikilink topology)
│   └── enrich.py       # LLM-powered enrichment (person cache, suggestions)
├── providers/
│   ├── gitlab.py       # glab CLI wrapper
│   └── github.py       # gh CLI wrapper
└── index/
    ├── embedder.py     # chunk notes, embed via nomic ONNX (LRU-cached queries)
    ├── store.py        # LanceDB: upsert, delete, hybrid/keyword/vector search
    ├── router.py       # adaptive query classification
    ├── corrective.py   # retrieval quality check + query reformulation
    ├── graph_rag.py    # wikilink-based search expansion (cache-aware)
    ├── hyde.py         # hypothetical document embeddings
    ├── contextual.py   # chunk context prepending
    ├── late_chunking.py # full-doc embedding with per-chunk pooling
    ├── chunking.py     # pluggable chunking strategies
    ├── models.py       # embedding model registry
    ├── reindex.py      # full/incremental rebuild
    └── health.py       # index health tracking

Testing approach

• Unit tests mock alaya.zk.run_zk (the subprocess boundary) and the embedding model. No subprocess calls, no model loading. Obsidian tests use real filesystem fixtures.
• Wiring tests prove each tool dispatches through the backend by patching run_zk with AssertionError side effects.
• Integration tests use the real zk binary and LanceDB against vault_fixture/.
• Tests use a copy of vault_fixture/ in a temp directory — vault is never modified in place.
• TDD: failing tests written before implementation for every tool.

Companion: alaya-tui

alaya-tui is a Go + Bubble Tea terminal dashboard for vault observability and agent-agnostic chat. It shows vault health, tails the audit log, browses notes, and spawns any configured LLM agent as a subprocess.

Status

All 5 milestones implemented plus advanced RAG pipeline, pluggable vault backend (zk + Obsidian), and in-memory metadata cache. 607+ unit tests passing. See open issues for planned improvements.