Status: normative E8 deployment reference
Issue: #675
Parent: #667
Related: #670 #673 #674 #750
Architecture: Enterprise Control Plane
This specification freezes the boundary for the optional always-on IDKMesh control service used by G2 and G3 deployments.
It does not make an always-on service the default product path. G0/G1 remain GitHub-first and server-optional. A team should introduce G2/G3 only when private-network access, tenant isolation, persistent coordination, operational SLOs, centralized audit, or similar requirements cannot be met cleanly by the GitHub-first profile.
This document also does not select Kubernetes, PostgreSQL, Redis, a cloud provider, an ingress product, or a secret manager. Those are deployment choices behind the boundary defined here.
Normative terms such as MUST, MUST NOT, SHOULD, and MAY describe the deployment contract. They are not claims that a production G2/G3 service is already deployed or independently validated.
Moving coordination into an always-on service MUST NOT move repository integration authority into that service.
The invariant is:
control service can admit/route/coordinate
worker can execute and produce a candidate
verifier can produce evidence/recommendation
human/governance can record a decision
GitHub protected integration remains canonical
Unless a future reviewed ADR explicitly changes the model:
G2/G3 deployments MUST preserve separate authorities even when several components share one physical cluster.
| Plane | Owns | Must not own |
|---|---|---|
| API / ingress | authentication handoff, request normalization, tenant context, rate/size limits, request correlation | provider execution, direct repository merge |
| Control / application | admission, authorization orchestration, idempotency, claims/leases, connector dispatch intent, run state | verifier truth, final integration |
| Worker / execution | bounded execution against an admitted WorkUnit | control DB administration, policy mutation, merge |
| Data | durable run/event/idempotency/decision records and immutable evidence references | implicit authorization decisions |
| Evidence / audit | verification evidence, privileged-event audit, export/checkpoints | execution or integration authority |
| Source / integration | repositories, PRs, rulesets, protected integration | worker execution authority |
A small G2 deployment MAY colocate processes for operational simplicity, but the logical boundaries, identities, permissions, and data access rules above still apply.
Every product/resource request MUST establish the following context before tenant state is read, mutated, dispatched, or materialized:
authenticated transport/principal
-> normalized ActorContext
-> explicit TenantScope / project scope
-> exact resource reference
-> AuthorizationDecision
-> idempotency/concurrency check
-> durable intent/state transition
-> bounded side effect
-> audit/evidence record
The authorization decision MUST bind the exact tenant/project and resource scope. Global role membership alone is insufficient.
For G3:
/healthz and /readyz are operational endpoints rather than tenant
resource requests. They MAY be reachable without tenant context, but they MUST
return no tenant/project/run/evidence/identity/secret data.
The service consumes the identity/authorization owner defined by Enterprise Authorization Kernel v0.1; E8 does not create a second role model.
API replicas SHOULD be stateless wherever practical.
Canonical coordination state MUST NOT exist only in:
Durable state is owned by an explicit database/ledger/object boundary according to API Storage Profiles v0.1.
At minimum the network profile must have durable ownership for:
Raw credentials MUST NOT be stored in API/control state.
A G2/G3 control service MUST remain correct with more than one API/control replica before it is described as horizontally scalable.
Required semantics:
not_submitted, submission_unknown, and provider_accepted;Scaling throughput is not allowed to weaken separation of duties or create a second integration authority.
The reference G2/G3 worker pattern is outbound/bounded:
private/self-hosted runner
-> receives one admitted WorkUnit / execution binding
-> resolves only authorized short-lived secret material
-> executes in its bounded substrate
-> returns candidate artifacts + provenance/evidence references
-> loses the temporary execution authority
Workers SHOULD be reachable through a private network, brokered job channel, or outbound polling model appropriate to the deployment.
Workers MUST NOT require:
A self-hosted runner may create a candidate branch/PR only through a narrowly scoped, audited integration adapter when project policy permits it. Candidate publication is not protected integration.
Every G2/G3 service deployment MUST expose separate operational probes:
GET /healthzProcess/container liveness only. It answers whether the service process is capable of serving requests. It MUST NOT perform expensive dependency queries or expose tenant state.
GET /readyzTraffic readiness. It MUST fail when the instance cannot safely serve its declared profile, including incompatible schema/migration state or unavailable dependencies required for correctness.
Readiness MAY include coarse dependency states such as database/audit/identity adapter readiness, but MUST NOT expose secrets, tenant identifiers, raw errors, prompts, or evidence.
A dependency outage must map to a documented degraded/fail-closed mode. For example, losing an optional SIEM exporter may preserve bounded coordination while retaining audit locally, but losing the authoritative durable store must make mutation readiness fail.
Schema migration is an explicit deployment phase.
Requirements:
The deployment MUST define backup ownership for every durable component required to reconstruct control state.
The backup set includes, as applicable:
Backup copies MUST be encrypted according to the deployment data policy. Retention, RPO, and RTO are deployment/profile fields, not universal product claims.
Restore procedure MUST specify:
Observed restore-time and data-loss evidence belongs to E6/E9. This E8 specification defines the boundary; it does not manufacture a successful DR exercise.
For API/application changes compatible with the current durable schema:
No admitted request may be silently abandoned merely to make deployment finish.
Use the expand/migrate/contract sequence in section 9. Mixed-version operation is permitted only inside an explicitly tested compatibility window.
Use bounded downtime:
Rollback MUST NOT start old code against a newly incompatible schema. Recovery must be a supported down-migration, roll-forward, or restore operation.
The reference is technology-neutral:
GitHub / protected integration
^
| candidate PRs, checks, decisions
|
enterprise IdP -> private/reviewed ingress
|
+-------+-------+
| API replicas | stateless where practical
+-------+-------+
|
tenant + authorization
|
+-------+-------+
| control svc | admission/idempotency/dispatch
+---+-------+---+
| |
durable DB/ledger +------> audit/export sink
|
immutable evidence refs
|
+--------+---------+
| |
hosted connectors private/self-hosted runners
| |
+--------+---------+
|
candidate/evidence
The data plane is not exposed directly to workers. The execution plane is not used as a durable database. GitHub remains the source/integration plane.
A G2 deployment may run one replica when its declared availability objective permits that. A G3 deployment should normally run multiple failure-isolated API replicas and a durable state layer suitable for the declared SLO/RPO/RTO, but replica counts are deployment policy rather than hard-coded product constants.
| Requirement | G2 | G3 |
|---|---|---|
| Always-on service | required | required |
| Tenant/project scope | explicit | mandatory and isolation-critical |
| Multi-tenant shared state | not required | supported only with proven isolation |
| Per-request authorization | required | required with tenant/resource binding |
| Durable coordination state | required | required |
| Audit | required for privileged actions | tenant-scoped and tamper-evident |
| Worker merge authority | forbidden | forbidden |
| Horizontal scale semantics | defined even if deployed single-replica | required before shared-service scale claim |
| Backup/restore lifecycle | defined and exercised per E6 profile | defined and exercised per E6 profile |
| GitHub canonical integration | yes | yes |
A G2 service MUST NOT be renamed G3 merely because it hosts several projects. G3 requires the tenant-isolation evidence owned by E2/E3/E9.
Before enabling G2 mutations/dispatch:
Before claiming G3, additionally verify:
| #675 acceptance criterion | Frozen contract |
|---|---|
| API/worker/data planes separated | sections 3 and 12 |
| stateless API where practical; durable explicit DB/ledger | section 5 |
| per-tenant authorization at every request | section 4 |
| no direct merge authority in worker services | sections 2, 3, and 7 |
| private-network/self-hosted runner pattern | section 7 |
| horizontal scale/idempotency semantics | section 6 |
| migrations/backups/health/readiness defined | sections 8-10 |
| zero- or bounded-downtime upgrade strategy | section 11 |
| GitHub remains canonical code/integration authority | sections 2 and 12 |
This mapping means the E8 boundary/reference is specified. It does not mean a particular deployment has demonstrated these properties.
This specification does not:
Operational proof remains evidence-gated:
The service boundary is therefore complete as an architectural contract while production deployment claims remain conditional on those observed controls.