Skip to content

DDR-007 — Relevance over time is computed, never stored

Version: 0.1.0 | Date: 2026-09-24 | Status: Accepted

Context

DDR-003 Phase D introduced confidence decay. Today it is a batch job, engrama decay, available only through the CLI, and it rewrites stored data:

confidence ← confidence · exp(−rate · days_since(updated_at))

The job rewrites confidence but leaves updated_at untouched, so each run re-applies the whole interval since the last real update. The decay compounds. After k daily runs a node untouched for k days holds exp(−rate · k(k+1)/2) instead of exp(−rate · k). With the default rate = 0.01, a node untouched for a month drops to 0.0096 instead of 0.74. Measured on a SQLite store, three consecutive runs on a 10-day-old node yield 0.905, 0.819 and 0.741, where the intended value is 0.905 every time.

The rest of the pipeline amplifies that error:

  • Nothing restores confidence. merge_node sets confidence only ON CREATE, so a node that is actively updated stays at its decayed value. Reflect regenerates the same Insight titles, and those Insights keep their decayed confidence, so a "freshly generated" Insight can report a confidence close to zero.
  • Archiving is blind. Nodes below --min-confidence are archived whatever their degree or type, and archiving sets updated_at = now, so the archived node looks fresh to every temporal signal. Well connected hubs get archived like any other node, and live nodes whose only path to the rest of the graph ran through them are cut off. Once archived, a node disappears from fulltext search, but inline relation lookup can still target it.
  • The job has no scope. It sweeps every tenant at once, and it applies equally to Insights, Domain nodes and hubs.
  • Import loses history. engrama import replays nodes through merge_node, which drops created_at and updated_at (#76). An export/import round trip dates every node to the day of the import, so any recency signal computed afterwards is wrong.

A working memory that spans months needs recency to rank results, not to delete them.

Decision

confidence means belief in the fact. The writer sets it, and it changes only with an explicit statement (supersession via valid_to, or a correction). Recency is a read-time ranking signal computed from activity. Nothing decays in storage, and nothing is archived automatically.

Concretely

  1. last_activity_at. merge_node sets it on the node. merge_relation sets it on both endpoints, so a hub that keeps gaining edges stays recent even when its own properties never change. It is backfilled from updated_at on first use.
  2. Read-time recency. temporal_score in core/search.py becomes 2^(−age / half_life), where age is measured from last_activity_at. The half-life depends on the kind of node:
  3. domain nodes: 180 days;
  4. system Insights (DDR-006 item 7): 30 days;
  5. Domain and anchor labels: exempt (score 1). A degree factor stretches the half-life of hubs: half_life · (1 + log2(1 + degree)). Every parameter is configurable (ENGRAMA_RECENCY_*) and reported by engrama_status.
  6. engrama decay is deprecated. It changes nothing: it accepts the same flags, prints a deprecation notice that points at read-time recency, and exits cleanly, so existing scripts keep working. It stays for one minor release and is then removed, together with the decay_scores / decay_confidence store methods.
  7. Archiving is always explicit. Only an explicit forget or an approved curation Insight (archive_candidate, DDR-008) archives a node. Archiving no longer touches updated_at, and it records archived_at and archived_reason. When an archived node still bridges live nodes, the curation Insight names the bridge.
  8. Reflect confidence refreshes. Regenerating an Insight updates confidence and body ON MATCH (its evidence may have changed), unless a human already judged it (approved or dismissed).
  9. Import preserves timestamps. The importer passes created_at / updated_at through a trusted path. The caller-supplied property bag remains untrusted, so #76 still holds for MCP and SDK writes.

Consequences

  • Positive:
  • Recency can no longer destroy data: a bad parameter only produces a bad ranking, and changing the parameter fixes it retroactively.
  • No cron is needed. Nothing needs scoping for multi-tenant deployments either, because the signal is computed per query inside the caller's scope.
  • Hubs and actively linked nodes stay visible.
  • Negative:
  • Graphs that were already decayed keep their low stored confidence, because this DDR does not rewrite existing data. Operators who want a reset can set confidence = 1.0 where valid_to IS NULL with a documented one-liner; there is no automatic migration.
  • Search does slightly more work per result, one exponential per candidate, which is negligible next to retrieval.
  • Neutral:
  • engrama decay output changes meaning (report instead of write) and is announced in the changelog.

Supersedes / Superseded by

Supersedes the persisted-decay part of DDR-003 Phase D. Temporal validity (valid_from / valid_to, revival) is unchanged.

Open follow-ups (not blocking)

  • Neighbour-activity propagation (a hub inherits a fraction of its neighbours' recency). Deferred until the degree factor has been measured on real graphs.
  • Retention policies (hard deletion after N months) for deployments with legal retention limits. That is a separate concern from relevance.

References

  • DDR-003: temporal reasoning (Phase D).
  • DDR-008: curation Insights, which replace automatic archiving.

  • 76: server-managed timestamps.