Skip to content

DDR-005 — Opt-in authentication for the Streamable HTTP transport

Version: 0.1.0 | Date: 2026-09-16 | Status: Proposed

Context

Engrama has shipped a Streamable HTTP transport since 0.12, and since 0.13.0 every node and relation is owned by an (org_id, user_id) identity with fail-closed reads. resolve_scope() in engrama/adapters/mcp/server.py derives that identity per request from the X-Engrama-Org-Id / X-Engrama-User-Id headers, and ENGRAMA_REQUIRE_IDENTITY makes a header-less request fail closed instead of pooling into the standalone identity.

What is missing is the other half: nothing authenticates the caller. The transport validates Origin and Host for DNS-rebinding protection (engrama/adapters/mcp/http.py) and stops there. The identity headers are therefore trusted input — correct when an authenticating gateway is the only thing that can reach the server, and dangerous in any other topology, because a caller who can reach /mcp can name any identity it likes.

That leaves remote self-hosting as "the operator's problem", which is where #121 comes in: an EU platform offered to publish a one-click deploy recipe for Engrama. The honest answer was that a one-click deploy of an unauthenticated memory server is a footgun, and that putting a single shared static bearer token at the edge would make that token the entire authentication layer of somebody's long-term memory. The same request keeps arriving in other shapes — someone wanting a private remote instance for a phone client, a team wanting one agent host reachable from several machines.

The groundwork is already in the tree, deliberately: --auth-issuer / ENGRAMA_AUTH_ISSUER is already parsed, and /.well-known/oauth-protected-resource (RFC 9728) already returns 404 until an issuer is configured, with a comment promising that "the auth phase only has to set an env var, not touch code". This DDR is that phase.

Two constraints shape the design. Authentication must not become a dependency, a configuration step or a behaviour change for the majority of installs, which are single-user and local — so everything here is opt-in and inert by default. And Engrama must not grow into an identity provider: verifying a token is in scope, issuing one is not.

Decision

Add two opt-in authentication modes to the Streamable HTTP transport, both disabled by default, and make the identity headers untrusted whenever either one is active. The stdio transport is unaffected.

Concretely

  1. OAuth 2.1 resource-server mode. engrama/adapters/mcp/auth.py validates an RS256 bearer JWT against a configured issuer: signature (JWKS, lazily fetched and cached by kid), iss, aud, exp, and optionally a set of required scopes. Engrama is a pure resource server — it mints nothing, registers no clients, and implements no flows. Bring your own IdP (Keycloak, Authentik, Auth0, Entra, …).

  2. New configuration, all unset by default.

  3. ENGRAMA_AUTH_ISSUER — already parsed today; setting it now also turns on token validation, not just discovery metadata.
  4. ENGRAMA_AUTH_AUDIENCE — required when the issuer is set.
  5. ENGRAMA_AUTH_JWKS_URI — optional override; derived from the issuer's OIDC metadata when absent.
  6. ENGRAMA_AUTH_SCOPES — optional CSV of required scopes.

  7. ENGRAMA_AUTH_ORG_CLAIM — optional claim naming the caller's organisation.

  8. Claims → identity. user_id is always the sub claim, the per-user isolation boundary. org_id is the configured org claim when set and present (first entry if the claim is a list), else sub, so a single-user deployment gets org_id == user_id with no configuration. A token that would resolve either value to the reserved __entity__ org-shared sentinel is rejected outright: a token must never be able to name the org-shared scope.

  9. Header trust inverts when auth is on. With authentication enabled, inbound X-Engrama-* identity headers are stripped before the request reaches the tools, and replaced with the pair derived from the validated claims. There is exactly one authority for identity per deployment: the token when auth is on, the fronting gateway when it is off. They are never mixed, because accepting both would let any caller forge another tenant's scope by adding a header.

  10. Static service-token mode. ENGRAMA_AUTH_TOKEN gates the endpoint on a single shared secret, compared in constant time, for single-tenant or ephemeral deployments where every caller is the same principal. It maps to one stable identity (ENGRAMA_AUTH_TOKEN_SUB, defaulting to the standalone identity). This mode is honest about what it is: one principal, no per-user isolation, no revocation short of rotating the secret. It is mutually exclusive with the OAuth mode — configuring both fails fast at startup rather than leaving the precedence ambiguous.

  11. Discovery and challenges. With an issuer configured, /.well-known/oauth-protected-resource serves the RFC 9728 document it already knows how to build, and a rejected request answers 401 with WWW-Authenticate: Bearer resource_metadata="<url>", so an MCP client can discover where to authenticate instead of guessing. /health and /.well-known/* remain unauthenticated.

  12. Optional dependency. Token validation lives behind a new engrama[auth] extra (PyJWT + cryptography). The base install stays dependency-light, and a deployment that configures an issuer without the extra fails at startup with an actionable message.

  13. Failure mode. Misconfiguration fails closed and loud at startup — an issuer without an audience, an unreachable JWKS at first use, both modes configured at once. A server that cannot validate tokens must not start serving /mcp unauthenticated.

Consequences

  • Positive. Remote self-hosting becomes a supported configuration instead of a documented hazard, which unblocks platform deploy recipes (#121) and every "I want my own private instance" request behind them. The security docs stop having to say "ships without authentication — keep it on loopback". Defence in depth improves even for deployments that already sit behind a gateway, since the inner server can verify the same tokens rather than trusting injected headers.

  • Negative. The project now ships security-critical code: a bug in token validation is a vulnerability, not an inconvenience, and it raises the bar for reviews and releases in this area permanently. The configuration surface grows by six variables, and the deployment matrix grows with it: unauthenticated, token, and OAuth are three paths that all need covering in tests.

  • Neutral. Default behaviour is unchanged: no issuer and no token means today's server, byte for byte, and stdio never consults any of this. Deployments that authenticate at a gateway and inject identity headers keep working exactly as they do now, because with auth disabled the header path is untouched; migrating them to token verification is optional and independent of this change.

Open follow-ups (not blocking)

  • Opaque-token introspection (RFC 7662) for issuers that do not use JWTs.
  • Per-tool scope mapping, so a token can be read-only across the graph rather than all-or-nothing at the endpoint.
  • Rate limiting and an audit trail for authentication failures; today a brute-force attempt is invisible beyond ordinary logs.
  • mTLS as an alternative to bearer tokens for machine-to-machine deployments.
  • Dynamic client registration and token issuance stay out of scope permanently. Engrama is a resource server; anything that mints tokens belongs in an identity provider or a gateway in front of it.

References