IDKMesh

IDKMesh API Control-Plane Architecture

Status: planning baseline
Date: 2026-09-23
Program: #713
Companion plan: API_V1_PROFESSIONALIZATION_PLAN_2026-09-23.md
Cross-cutting conventions: every endpoint under /api/v1 follows the frozen API Conventions v0.1 (error envelope, status mapping, pagination, idempotency, concurrency, deprecation — ADR-0018). This document defines how surfaces compose; the conventions specification defines how each one talks HTTP.

Purpose

This document defines how IDKMesh API surfaces fit together so the Control Tower, connector control plane, Product Spine, CLI, and future multi-user service do not evolve into separate systems with different semantics.

Architectural rule

one domain model
+ one application-service layer
+ multiple transport/client adapters

HTTP, CLI, GitHub Actions, browser UI, and provider connectors are adapters. They may transform transport details but may not redefine WorkUnits, runs, evidence, human decisions, or authority.

Logical layers

1. Domain contracts

Canonical domain objects remain independently versioned:

No HTTP endpoint may create a competing replacement object merely for UI convenience.

2. Application services

Application services own use-case semantics:

They are provider-neutral and transport-neutral.

3. Policy / authority boundary

Before a state-changing application service executes, it receives explicit:

A service may return requires_approval; it must not invent missing approval.

4. Persistence

Persistence adapters retain:

Persistence stores decisions/evidence; it does not grant authority.

5. Transport adapters

Profiles:

Local HTTP

Network HTTP

CLI

Calls the same application services. CLI flags are not a parallel policy model.

GitHub-native adapters

GitHub events/actions are untrusted inputs until normalized and authorized. Labels/comments are projections or triggers, not canonical authority.

Namespace

Product HTTP resources converge on:

/api/v1

Runtime probes remain outside the product namespace:

Object schema versions remain independent from URL compatibility versions.

Dependency diagram

The architectural rule above, grounded in the modules that exist on main today (not a target-state sketch):

+----------------------------------------------------------------+
| Domain contracts (schemas/*.schema.json)                       |
| WorkUnit, RoutingDecision, CandidateReference, ResultManifest,  |
| VerificationResult, Run Evidence Report, Human Decision Record  |
+----------------------------------------------------------------+
                              ^
                              | validated against, never redefined by
                              |
+----------------------------------------------------------------+
| Application services (idkmesh/*.py)                            |
|  control_tower_api.py     -- read-only evidence inspection      |
|  connector_store.py       -- connections/routing/run persistence|
|  connector_routing.py     -- RoutingDecision resolution         |
|  product_spine_run_store.py -- run create/status/cancel/events  |
|  local_loop.py            -- WorkUnit -> attempts -> evidence   |
|                               (delegates to experiments/*)      |
+----------------------------------------------------------------+
          ^                 ^                  ^              ^
          | thin dispatch   | thin dispatch    | thin dispatch |
          |                 |                  |               |
+---------+------+  +-------+--------+  +------+-------+  +----+----+
| Local HTTP     |  | CLI            |  | GitHub-native |  | Network |
| control-tower  |  | idkmesh {...}  |  | Actions/hooks |  | HTTP    |
| /api/v1, read  |  | connections,   |  | untrusted     |  | (not    |
| -only + GET    |  | run, local-loop|  | until checked |  | built)  |
| runs, GET runs/|  |                |  |               |  |         |
| {id}, GET runs/|  |                |  |               |  |         |
| {id}/attempts, |  |                |  |               |  |         |
| GET runs/{id}/ |  |                |  |               |  |         |
| evidence,      |  |                |  |               |  |         |
| GET work-units,|  |                |  |               |  |         |
| GET work-units/|  |                |  |               |  |         |
| {id}, GET      |  |                |  |               |  |         |
| projects/{id}, |  |                |  |               |  |         |
| GET events,    |  |                |  |               |  |         |
| GET events/    |  |                |  |               |  |         |
| stream (#739,  |  |                |  |               |  |         |
| #741)          |  |                |  |               |  |         |
+----------------+  +----------------+  +---------------+  +---------+

The Local HTTP box’s run, WorkUnit, and project read endpoints are opt-in per server instance (idkmesh control-tower --product-spine-store PATH); they return 503 without it, rather than being conditionally present in this diagram. The WorkUnit and project endpoints are read models derived from stored runs (ADR-0021), not a WorkUnit or project store; idkmesh work-unit list|status and idkmesh project status read through the same service.

The Local HTTP box is a stdlib development server with bounded limits (ADR-0022): a 10 s request timeout, a 16-request concurrency cap that answers 503 + Retry-After instead of queueing, graceful drain, and the stdlib parser bounds (request line 65536 bytes, 99 header fields, 2 MiB body), all published at operations.limits on GET /api/v1/status. These bound request handling, not accepted connections, and 429 is not implemented (no per-client identity yet). SSE streams have their own client limit, heartbeat and maximum lifetime, published in the same object. The stdlib server must never face the Internet: a network or multi-user deployment requires the reviewed production transport adapter, which is the Network HTTP box and is not built.

The Local HTTP box’s GET /api/v1/events and GET /api/v1/events/stream read the canonical event source (ADR-0023): an append-only events table in the same SQLite store (store schema version 2), written in the same transaction as the run change it records, with a monotonic sequence, producer-supplied occurred_at, and a Last-Event-ID-resumable, bounded, poll-based SSE stream (at-least-once; dedupe on event_id). Coverage is deliberately narrow: only Product Spine run create and run cancel emit events (run.created, run.cancelled); the offline idempotent spine and the GitHub dispatch/status writers do not yet, nothing prunes the table, and WebSocket is deferred. idkmesh events list reads through the same service. A store opened by code that predates schema version 2 is refused by the existing “newer than supported” check rather than silently ignoring events.

The Local HTTP box’s GET /api/v1/runs/{run_id}/evidence serves the Run Evidence Report that the idempotent offline spine retains in the run row (ADR-0024). It re-verifies canonical_digest(report) against the run’s evidence_report_digest on every read and refuses to serve a mismatch (500 evidence_integrity_error). The run listing selects an explicit set of Product Spine row kinds, so the foreign rows that share the runs table (admission-only, execution-error and GitHub dispatch/status rows) cannot break it. GET /api/v1/runs/{run_id}/decisions is not built: no decision content is retained, and recording a decision needs an authenticated, accountable principal (issue #740, enterprise identity), which is a governance gate.

Every box in the bottom row is a transport/client adapter over the same application services; none may redefine a domain contract to fit its own transport. The Network HTTP adapter is explicitly not built yet – see “Architecture completion evidence” below.

Resource ownership map

Resource/use case Canonical owner
projects ProjectManifest/Product Spine
connections/probes Connector Control Plane (#570/#580)
WorkUnit preview Product Spine + connector service
routes RoutingDecision / connector kernel
runs Product Spine / run service
attempts/candidates run service + CandidateReference
result manifests ResultManifest contract
verification verifier/evidence service
run evidence Run Evidence Report
human decisions Human Decision Record
resource/compute admission zero-project-spend compute router (experiments/free_compute_router.py, scripts/free_resource_planner.py, ADR-0006)
events canonical event service (#741)
integration execution separate protected integration authority

Read versus mutation separation

Read surfaces may aggregate projections, but must expose the underlying provenance/digests.

Mutation surfaces must add:

Human Decision boundary

The Human Decision API records:

who decided
what decision
why
which exact evidence digest
when

It does not:

A later integration actor must independently consume the decision plus policy.

Event architecture

Events are append-only facts about state transitions, not a replacement for the domain records they reference.

The future event service must support:

SSE is the first live-delivery transport because current needs are one-way observation.

Local-to-network evolution

The local API is not thrown away when network mode arrives.

Shared:

Replaced/extended:

Failure model

The API distinguishes:

A worker/provider failure is a domain outcome and should not automatically be reported as an HTTP server failure when the request itself was processed correctly.

Security model

Local

The session token protects the local browser/API boundary. It is not a user identity.

Network

Principal identity comes from a trusted authentication adapter and is evaluated through #670 policy semantics. Role/authority is never inferred from model output, issue text, labels, or self-declared headers.

Operational model

All HTTP services should converge on the reusable runtime baseline:

Later network services add:

Implementation ownership

The API program does not absorb existing subsystems.

Anti-patterns

Reject designs that:

Architecture completion evidence

This architecture is considered implemented only when:

  1. Control Tower and connector endpoints share the conventions/application service boundary;
  2. domain objects are schema-bound;
  3. mutations enforce identity/policy/idempotency;
  4. persistence and events are restart-safe;
  5. a client can follow WorkUnit -> run -> evidence -> human decision without provider-specific knowledge;
  6. integration remains separately authorized.

Verified against current main – 2026-09-27

Not yet fully implemented; measured state per criterion, so a later pass can tell what actually changed rather than re-deriving all six from scratch:

  1. Partial. idkmesh/control_tower_api.py emits the frozen conventions (idkmesh-api-error, service headers) over real local HTTP endpoints. No connector-control HTTP endpoint exists yet to compare against – docs/specifications/CONNECTOR_CONTROL_API_V0_1.md remains an unimplemented design contract (issue #580). Not measurable until a connector HTTP endpoint ships.
  2. Partial. Domain contracts (WorkUnit, ResultManifest, VerificationResult, Run Evidence Report, Human Decision Record) and the cross-cutting envelopes (error, list, status, inspection, readiness – issue #736/#737) are schema-bound. The event envelope and a human-decision API request/response wrapper are not (issue #737 remains open for both).
  3. True for the CLI mutation path. idkmesh run create --idempotency-key persists to a SQLite idempotency_key TEXT ... UNIQUE constraint in idkmesh/connector_store.py, so a duplicate key with a different request digest fails closed rather than double-creating a run. No HTTP mutation endpoint exists yet to verify the same at the transport layer.
  4. Partial. Product Spine run/connection state persists in a restart-safe local SQLite store (idkmesh/connector_store.py). The canonical event service (#741) referenced by the ownership map above does not exist yet, so “events are restart-safe” is not yet measurable.
  5. True for one bounded case, via the CLI. idkmesh local-loop <config> (issue #883/ROADMAP S4 R1) runs WorkUnit -> two isolated attempts -> independent verification -> evidence report end to end without the caller naming a worker/provider. Recording a human decision and replay are still separate, deliberately manual commands, not part of this client path. Not yet exposed over HTTP.
  6. True. No code path in idkmesh/control_tower_api.py, idkmesh/local_loop.py, or the Product Spine CLI grants merge/push authority; idkmesh local-loop explicitly prints next steps (record_human_decision.py, replay_run.py) rather than executing them.

Net: this architecture is grounded and consistently followed where it has been implemented, but is not yet complete by its own six-criterion bar – criteria 1 and 4 are blocked on work (a connector HTTP endpoint, the event service) that does not exist yet, not on a design disagreement.