Skip to content

DDR-006 — The engine, not the agent, keeps the graph connected

Version: 0.1.0 | Date: 2026-09-24 | Status: Accepted

Context

Engrama's write contract relies on agent discipline. The system prompt (v0.5/v0.6) and the engrama_relate docstring say "call relate right after remember", and engrama_remember answers relations_created: 0 without further comment when the agent does not. Sustained use shows what happens when an agent forgets, loses context, or reaches for a tag instead of an edge. The graph degrades in recognisable ways:

  • Tags instead of edges. Nodes carry a tag that is literally the name of an existing anchor node (a project, a course, a domain) but have no edge to it. Search still finds them, but the graph does not connect them, and a large share of the live nodes can end up with no edge at all.
  • Stubs that become hubs. Inline relations create a target stub (status: stub) when nothing matches. Nothing ever clears that status or asks for enrichment, so a stub can collect many edges while having no summary. Those stubs carry real structure: removing them splits the graph, so the fix is to enrich them, not to drop them.
  • Duplicates across labels and names. lookup_node_label matches a name case-insensitively on any label and returns an arbitrary one (LIMIT 1), while merge_node is label-specific, so the same entity can exist under two labels. Most duplication is semantic rather than lexical (a product, "the product's API" and "the product's certificate" as three unlinked nodes), so exact name matching finds few of them.
  • Label and status drift. Labels get stretched past their meaning (Domain, meant for areas of knowledge, used for internet domain names). status is free text in practice, including values in the user's own language, so the queries that filter on open / active quietly miss nodes.
  • Two meanings of Insight. Reflect writes Insight nodes for review, and agents also use the label, correctly, for syntheses they write themselves: lessons learned, post-mortems, conclusions. Those hand-written Insights tend to be well documented and well linked, but every detector and structural query skips :Insight, so they never take part in curation.

The fuzzy resolution added in #93 is a good start but has three limits. It is lexical only (difflib). It never runs for the node being remembered, only for inline targets. It scans an alphabetically ordered, capped list of candidates (list_existing_nodes, LIMIT 1000), so above 1000 nodes the tail of the alphabet is never a candidate. The scan is also linear: in a synthetic benchmark it takes 13 ms per write at 1 000 nodes and 300 ms at 20 000. It also resolves to archived nodes, which search hides.

Decision

Every write goes through one entity resolver, and every write response tells the agent the structural consequence of what it just did. The engine guarantees resolution, feedback and stub hygiene regardless of how careful the agent is. Rejection stays opt-in.

Concretely

  1. Entity resolver (engrama/core/resolve.py), used by engrama_remember for the node itself and for every inline target. The policy is pure and identical on every backend; the stores only supply bounded, scope-filtered candidates:
  2. name_candidates: nodes whose name contains a three-character fragment of each word of the normalised target (casefold, accents stripped, -/_/space unified), exact names and similar lengths first, capped at 200. Unlike a fulltext lookup, this still finds a candidate when the target has a typo;
  3. vector kNN, for the remembered node only, reusing the embedding already computed for embed-on-write. Neighbours are re-scored with an exact cosine, because each backend's vector index reports similarity on a different scale.

Names are compared after normalisation (difflib ratio). The three-way policy from #93 (connect / ask / create) stays, with thresholds configurable (ENGRAMA_RESOLVE_*). Only near-certain lexical matches connect automatically; semantic similarity only ever warns. Archived nodes are never linked silently: an exact match on an archived node is reported as revive_candidate. 2. Resolution of the remembered node itself. When a node with a different key (or the same key under another label) resolves above the threshold, the write still happens. The response carries possible_duplicate_of: [{label, name, score, reason}]. ENGRAMA_REMEMBER_DEDUPE=off|warn|block sets the behaviour, with warn as the default. block returns an error with did_you_mean unless the call passes force_new: true. 3. Label-aware lookup. Exact matches prefer the requested label. An exact match under a different label is reported as label_conflict instead of silently picking one (replaces LIMIT 1). 4. Degree feedback. Every engrama_remember response carries the node's resulting degree. When it is 0 the response adds orphan_warning and suggested_relations, built from the resolver and from tag anchors (item 5). The opt-in ENGRAMA_REQUIRE_RELATIONS=true rejects degree-0 writes except for anchor labels (Project, Domain, Client, Course, Person). 5. Tag anchoring. A tag that resolves exactly (normalised or alias) to an anchor node in scope becomes a suggested relation, using a per-label default type (Project → BELONGS_TO, Course → COVERS reversed, Client → FOR, Domain → IN_DOMAIN). ENGRAMA_TAG_LINKING=off|suggest|auto, default suggest. With auto, the edges are created and listed in the response as relations_from_tags, so the caller sees exactly what was added. (Edges carry no properties in the SQLite backend, so they are not stamped.) 6. Stub lifecycle. A stub keeps status: stub only until it is enriched. A write that supplies summary or details clears it (to the caller's status, else active). Stub labels come from the explicit label, then the resolver, then the relation type. Relation-type inference is kept only as the last resort. Responses that touch a stub with degree ≥ 3 carry enrich_hint. 7. Controlled vocabularies. - status is validated against a per-label enum declared in the profile. Unknown values are normalised when a mapping exists; otherwise they are kept verbatim in status_note, with the status set to active and a warning. - Insight keeps both meanings, and they are told apart by origin. A system Insight is one that carries source_query, which reflect always sets. Only system Insights follow the review lifecycle and the Insight-specific rules in DDR-007 and DDR-008. A hand-written Insight is an ordinary domain node for the resolver, the detectors and the structural metrics. Writes are unchanged and nothing is relabelled. A single helper, is_system_insight(node), carries the rule for both backends. - The Domain label's description is tightened to "area of knowledge" in the profiles and tool docs.

Consequences

  • Positive:
  • The graph's structural quality stops depending on prompt adherence. An agent that has lost context still gets told about duplicates, orphans and the anchors it forgot to link.
  • The resolver replaces a linear, capped scan with index lookups, and measured candidate retrieval is flat in graph size: sqlite-vec kNN with scope takes 0.9 ms at 1 000 vectors and 5.3 ms at 5 000.
  • Negative:
  • Responses grow. Agents that pattern-match on the old response shape keep working, because the fields are additive, but they see more fields.
  • warn means some duplicates are still written. It is the default only because block needs a model-side retry loop.
  • Status normalisation rewrites values on the next write; the original is preserved in status_note.
  • Neutral:
  • The resolver runs with or without an embedder. Without one, it degrades to lexical resolution. The core never requires an LLM. A pluggable decider (EntityDecider protocol) can replace the threshold policy with a calibrated local classifier; that is an open follow-up.

Open follow-ups (not blocking)

  • A pluggable EntityDecider for resolution, relation-type choice and supersession, calibrated on real accept/reject data.
  • Deferred stub enrichment: a background task that drafts a summary for hub stubs from their neighbourhood. It needs an LLM, so it stays outside the core.
  • An aliases property consulted by the resolver. Aliases come from merging nodes, so they arrive with the merge operation (DDR-008).
  • Per-profile status vocabularies for the domain modules (hacking, teaching, photography, ai).
  • An episodic label (Event or Session) for session logs and periodic reports, which agents currently file as Insight for lack of a better fit.

References

  • DDR-001: faceted classification (labels and relation types).
  • DDR-003: embeddings and hybrid search, which the resolver reuses.
  • DDR-007: relevance over time. DDR-008: reflect as curator, which consumes possible_duplicate_of and the stub signals.

  • 93: confidence-based inline relation resolution.