CueCruxVaultCrux

Developer guide

Crux Daemon · 7. Connectors and sync

Crux Daemon developer guide

7. Connectors and scheduled work

Extensions are pull-based: something calls a tool. Connectors are push-based: the daemon wakes up on a cadence and does work. Both live in the same process and both write facts, but the machinery is different.

7.1 The scheduler

SyncScheduler (crates/corecruxd/src/sync_scheduler.rs:242) is one driver task for all periodic jobs. It sleeps to the earliest deadline rather than ticking every second.

let mut scheduler = SyncScheduler::new(fact_store.clone());
scheduler.register("my-job", Duration::from_secs(900), move || async { /* JobResult */ });
scheduler.spawn(shutdown_tx.subscribe());

(sync_scheduler.rs:264, main.rs:1491)

Contract you must know before registering anything:

  • The first attempt is one full interval after spawn, never at boot (sync_scheduler.rs:258). A restart storm cannot stampede an upstream.
  • A zero interval is clamped to one second. A job that should be off is simply not registered (sync_scheduler.rs:269).
  • The job closure must be re-callable.

JobResult — three outcomes, not two

(sync_scheduler.rs:80)

OutcomeStatus factBackoff
Ok(JobOutcome::Ran(detail))written, last_status: "ok"reset to zero failures
Ok(JobOutcome::Skipped(reason))not writtenuntouched
Err(message)written, last_status: "error", last_error setconsecutive_failures += 1

Skipped is the important one. A job that is inert this cycle — connector not connected, feature not configured — returns Skipped, which writes nothing and touches no state. An unconfigured daemon stays exactly as quiet as it was before the job existed (sync_scheduler.rs:33). Copy that discipline in your own jobs.

Backoff

The effective interval doubles per consecutive failure, capped at 4×, and resets on the first success (sync_scheduler.rs:154). The configured interval is never mutated — backoff is derived.

Status facts

One fact per job: entity __sync__::<job_id>, key status, schema crux.sync_job_status.v1 (sync_scheduler.rs:70).

{
  "schema": "crux.sync_job_status.v1",
  "job_id": "github-sync",
  "last_run_unix_ms": 1753440000000,
  "last_status": "ok",
  "last_error": null,
  "consecutive_failures": 0,
  "interval_secs": 900,
  "effective_interval_secs": 900,
  "next_run_unix_ms": 1753440900000,
  "detail": { "repos": 3, "commits_added": 12 }
}

__sync__:: is a born-private prefix, written with private: true and horizon_class: Volatile, actor sync-scheduler (sync_scheduler.rs:210). Node-local operational state is never push-eligible to a remote.

Read it with the ordinary facts routes — the scheduler deliberately adds no HTTP surface of its own (sync_scheduler.rs:24):

curl -s "http://127.0.0.1:14800/v1/facts/entity/__sync__::vault-watcher" \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN"

Persisting the status is best-effort: a store failure is warn-logged and never propagated (sync_scheduler.rs:225).

The two registered jobs

job_idIntervalRegistrationCondition
github-syncCORECRUXD_GITHUB_SYNC_INTERVAL_SECS, default 900main.rs:1506always registered; returns Skipped("github not connected") when idle
vault-watcherCORECRUXD_VAULT_WATCH_INTERVAL_SECS, default 300main.rs:1569only when the double gate passes

7.2 The markdown-vault watcher

This is the reference implementation of the file_watcher entry kind, and the one runtime that binds the pack model to daemon behaviour.

The double gate

Nothing runs unless both are true (vault_watcher.rs:18):

  1. A pack whose entry.kind is file_watcher is installed and granted on this node — checked via enabled_packs_of_kind. The first-party pack is vault.markdown-watcher.
  2. CORECRUXD_VAULT_WATCH_ROOTS names at least one absolute, readable directory.

activation() (vault_watcher.rs:193) returns one of three states, and half-configured is diagnosable from the boot log rather than from silence:

StateLog line
Active { pack_ids, roots }job registered
Inactivesilent — the ordinary default
HalfConfigured(reason)vault-watcher inactive with the reason (main.rs:1575)

The two half-configured messages are explicit about which half is missing (vault_watcher.rs:220):

file-watcher pack(s) ["vault.markdown-watcher"] are granted but
CORECRUXD_VAULT_WATCH_ROOTS is unset — nothing to watch; set it to
colon-separated absolute directories

CORECRUXD_VAULT_WATCH_ROOTS names 2 directories but no file-watcher
integration pack is installed+granted — grant `vault.markdown-watcher`
to activate

A corrupt integrations directory is treated as "no packs granted", not a boot panic (vault_watcher.rs:199).

Configuration

Env varDefaultMeaning
CORECRUXD_VAULT_WATCH_ROOTSunsetColon-separated absolute directories, PATH-style (vault_watcher.rs:105)
CORECRUXD_VAULT_WATCH_INTERVAL_SECS300Scan cadence; zero or unparseable falls back to the default (vault_watcher.rs:261)
CORECRUXD_VAULT_WATCH_TENANTlocalTenant the notes are sealed under
CORECRUXD_VAULT_WATCH_CORPUSdocsCorpus the notes are sealed under

parse_watch_roots (vault_watcher.rs:148) keeps only entries that are absolute, canonicalisable, and directories. Rejects are collected with a reason ((not absolute), (not readable), (not a directory)) and warn-logged once — operator intent is never silently dropped. Duplicates that canonicalise to the same path are deduped.

Walkthrough

# 1. Install and grant the first-party watcher pack.
curl -s -X POST http://127.0.0.1:14800/v1/console/integrations/vault.markdown-watcher/install \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN" \
  -H 'Content-Type: application/json' -d '{"version":"0.1.0"}'

curl -s -X POST http://127.0.0.1:14800/v1/console/integrations/vault.markdown-watcher/grant \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"version":"0.1.0","capabilities":["facts:read","facts:write"]}'

# 2. Point the daemon at your notes and restart it.
export CORECRUXD_VAULT_WATCH_ROOTS=/home/me/notes:/srv/shared/notes
export CORECRUXD_VAULT_WATCH_INTERVAL_SECS=120
export CORECRUXD_VAULT_WATCH_CORPUS=notes

# 3. After one interval, read the job status.
curl -s "http://127.0.0.1:14800/v1/facts/entity/__sync__::vault-watcher" \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN"

Step 2 requires a restart: activation() runs once during startup.

Scan safety properties

(vault_watcher.rs:284)

  • Hidden directories — anything whose name starts with ., which covers .git and editor state directories — are never descended; hidden files are skipped.
  • Symlinks are refused, files and directories alike. The walk uses symlink_metadata, which does not follow, so a link planted inside a vault cannot pull /etc into the corpus.
  • Every candidate is canonicalised and re-checked to be under the canonical root before it is read — belt and braces on top of the symlink refusal.
  • Only *.md (case-insensitive) is considered.
  • Files above MAX_FILE_BYTES (4 MiB) are skipped (vault_watcher.rs:121).
  • At most MAX_FILES_PER_CYCLE (500) files are ingested per cycle; the remainder is reported as pending and picked up next cycle (vault_watcher.rs:123).

Frontmatter

A leading --- line terminated by --- or ... is parsed as YAML (vault_watcher.rs:376). Two fields are understood: title (string) and tags (a YAML sequence, or an inline comma/space-separated string). A block that fails to parse yields empty frontmatter and is still stripped, so malformed YAML never leaks into the BM25 text. An unterminated block is treated as body, not frontmatter. A leading byte-order mark is tolerated.

The cursor

Second fact under the same entity: __sync__::vault-watcher key cursor, schema crux.vault_watcher.cursor.v1 (vault_watcher.rs:102).

{
  "schema": "crux.vault_watcher.cursor.v1",
  "updated_at_unix_ms": 1753440000000,
  "truncated": false,
  "entries": {
    "/vault/note.md": {
      "mtime_ms": 1753439000000,
      "size": 2048,
      "content_hash": "b3:9f86d0…",
      "seen_at_unix_ms": 1753440000000,
      "title": "Note",
      "tags": ["project", "crux"]
    }
  }
}

Change detection is two-stage (vault_watcher.rs:529): (mtime, size) decides whether to read the file at all; the content hash decides whether to re-seal it. A touch or an editor rewrite that leaves bytes identical refreshes the stat fields and skips the seal (vault_watcher.rs:679).

The serialised cursor is capped at 256 KiB. Over that, the most recently seen entries are kept, the rest are dropped, and truncated goes true. A dropped entry is re-ingested the next time it is seen — truncation costs work, never correctness (vault_watcher.rs:72).

Deletions are recorded, not applied

A note that disappears is removed from the cursor and listed in the status detail. No destructive store operation is performed — sealed segments are append-only and retracting a document needs a tombstone design that does not exist yet (vault_watcher.rs:77). The status detail says so explicitly in a deleted_note field.

Per-cycle report

JobOutcome::Ran carries this detail object (vault_watcher.rs:769): roots, tenant_id, corpus_id, scanned, added, modified, unchanged_content, deleted, deleted_note, pending, ingested_chunks, sealed_batches, cursor_entries, cursor_truncated, cursor_dropped, scan_errors, read_errors.

Failure semantics: all roots unreadable is an Err (drives backoff); some roots unreadable is reported in scan_errors and the cycle continues. A batch seal failure aborts the cycle, leaving those notes out of the cursor for retry (vault_watcher.rs:745).

Where the notes land

The watcher uses the same local prose ingest path as POST /v1/local/ingest (local_ingest.rs:685). Notes are chunked by chunk_markdown — split at ATX headings, then windowed at about 1800 characters with 180 characters of overlap, preferring paragraph boundaries (local_ingest.rs:598). doc_id is the canonical absolute path; chunk_id is <path>::<index padded to 6> (vault_watcher.rs:718).

Notes become BM25-searchable, and vector-searchable when a dense embedder is configured. tenant_id is the isolation key; corpus_id groups documents within it.

7.3 The GitHub connector

Nine routes (http/mod.rs:1275), handlers in http/integrations_github.rs:

RouteMethodScope
/v1/integrations/github/statusGETadmin:read
/v1/integrations/github/connectPOSTintegrations:install
/v1/integrations/github/disconnectPOSTintegrations:disable
/v1/integrations/github/syncPOSTintegrations:install
/v1/integrations/github/reposGETadmin:read
/v1/integrations/github/repos/accessibleGETadmin:read
/v1/integrations/github/repos/{owner}/{repo}/selectPOSTintegrations:install
/v1/integrations/github/repos/{owner}/{repo}/selectDELETEintegrations:disable
/v1/integrations/github/repos/{owner}/{repo}/planningPUTintegrations:install
# Connect once. The PAT is sealed immediately and never read back.
curl -s -X POST http://127.0.0.1:14800/v1/integrations/github/connect \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN" \
  -H 'Content-Type: application/json' -d '{"pat":"ghp_..."}'

# Choose repos, then sync on demand (the scheduler also runs it every 15 min).
curl -s -X POST http://127.0.0.1:14800/v1/integrations/github/repos/CueCrux/Crux/select \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN"
curl -s -X POST http://127.0.0.1:14800/v1/integrations/github/sync \
  -H "Authorization: Bearer $CRUX_AGENT_TOKEN"

run_sync_with_key (integrations_github_sync.rs:58) walks each selected repo and syncs four object types in order — commits, pull requests, issues, then issue and PR comments — writing facts with key record:

ObjectEntity
Commitgithub::{owner}/{repo}::commit/{sha}
Pull requestgithub::{owner}/{repo}::pr/{number}
Issuegithub::{owner}/{repo}::issue/{number}
Commentgithub::{owner}/{repo}::comment/{id}

Paging is capped at 100 per page and 10 pages per repo per sync — at most 1000 commits per repo per run (integrations_github_sync.rs:25). The cursor is last_synced_at_unix_ms per repo, converted to a since= parameter. Commits and comments are deduplicated by entity before writing; pull requests and issues are re-stored on each run.

Note that github:: is on the ungrantable prefix list (extension_grants.rs:120) — a community extension can never be granted read or write access to connector data.

7.4 The OpenAI-compatible connector

Five routes (http/mod.rs:1312), handlers in http/integrations_openai.rs: status (GET), connect (POST), disconnect (POST), settings (PATCH), and chat (POST).

chat proxies a chat-completions call, returning the upstream body plus a _proxy envelope. Model selection falls back body.model → the stored default_model → gpt-4o-mini (http/integrations_openai.rs:191). Calling it while disconnected returns 412 with detail OpenAI not connected; POST /v1/integrations/openai/connect first.

7.5 How connector credentials are protected

There is no generic secrets HTTP surface. Credentials are sealed envelopes written as fields inside per-connector JSON files.

Files: <data_dir>/integrations/github/credentials.json, <data_dir>/integrations/openai/credentials.json.

For your own connector: seal through encrypted_secrets::seal with state.integration_encryption_key (http/mod.rs:435), accept the secret over one POST, never echo it in a status response, and never write it to a file you construct yourself.

7.6 Building your own scheduled job

  1. Write the job body as an async closure returning JobResult. Return Skipped when unconfigured — never Ran with an empty summary.
  2. Register it with a stable job_id; that id becomes the public fact entity __sync__::<job_id> and is effectively an API.
  3. Gate activation on something explicit. The vault watcher's double gate is the pattern: a granted pack expresses "this may run here", an environment variable expresses "here is the input".
  4. Put a structured summary in Ran(Some(detail)). That is what an operator sees; it is the whole observability surface.
  5. Keep the body idempotent and re-callable. It will be retried on backoff.

Ground truth

More from Crux Daemon

WatchCrux

Independent read-only supervision monitoring confidence drift

OpsCrux

Operations control tower with live supervision and approval gating

AgentCrux

Coordinated multi-agent mesh with approval gates and budgets

Agent Principal Identity

First-class agent identity with activity history