rustybeds/wiki/10-modernization-roadmap.md

Modernization Roadmap

Purpose

This document defines how BEDS transitions from proof-of-concept broker plumbing to a modern, production-grade platform.

This is not a strict PHP feature port checklist. The objective is to preserve architectural strengths while replacing legacy mechanisms where better patterns now exist.

Product Direction

• BEDS is a product rewrite in Rust, not a language translation exercise.
• AMQP-first remains a core invariant.
• Service role is config-driven from one binary.
• The white-paper target is a Twitter-like backend showcase (SPEW) launched by configuration and template/domain declarations, with zero core framework code changes.

Historical Design Intent (Preserved)

These points come from operational experience in Namaste and are treated as architectural intent, not nostalgia.

1. Expensive resources should be long-lived:

• Logger transport clients, AMQP connections, and database clients should not be repeatedly instantiated for per-event semantic changes.

2. Template-derived operational state should live with runtime objects:

• The object that executes work should carry the template-derived state it needs for its full lifetime.

3. Public API contract must remain decoupled from internal schema:

• External developers interact with remapped/public field names and operation contracts.
• Internal table/collection names and DBO names remain private unless explicitly authorized.

4. Runtime template drift must be controlled:

• Running nodes should operate from a stable template snapshot for deterministic behavior.
• Template changes are applied through controlled reload/restart, not accidental file edits mid-run.

5. Template integrity is a future acceptance feature:

• Track per-template and registry-level fingerprints to detect drift and support cluster consistency checks.

6. Resource constraints are first-class requirements:

• BEDS must run on low-resource hardware (including RPi-class systems), so memory strategy must be intentional and bounded.

Current Program Mode

The project is currently in POC-first mode.

Immediate objective:

• Stand up two brokers (rBroker and wBroker)
• Attach them to AMQP service
• Prove message flow and broker replies

Guardrails are intentionally deferred until POC behavior is stable.

Implementation status update:

• Phase A transport stability evidence exists: live RabbitMQ round-trip tests for rec.read and rec.write ping paths.
• Phase B has started: REC template registry loading and startup validation are now implemented in IPL.
• Phase B progression: runtime template registry state is now persisted and passed into broker workers for dispatch-time template validation.
• Reliability progression: deterministic ack/nack behavior and retry/DLQ queue topology are implemented for rec.read and rec.write.

Must-Keep Invariants

1. AMQP-first data path for application operations.
2. Config-driven role selection (appServer, admin, segundo, tercero).
3. Template-driven domain onboarding.
4. Database-agnostic broker and factory boundaries.
5. DBA-owned schema boundary (no ad hoc query logic in app layer).

Modernization Requirements (First Pass)

Message Contracts

1. Define versioned request and response envelopes.
2. Standardize correlation and causation identifiers.
3. Validate payload shape at broker ingress with deterministic error responses.

Reliability Semantics

1. Define explicit ack and nack policy per error class.
2. Add retry strategy and dead-letter queue handling.
3. Add idempotency strategy for mutating operations.

Broker Runtime

1. Add supervised broker lifecycle (respawn failed tasks).
2. Add graceful shutdown and queue drain behavior.
3. Add health states and readiness semantics.

Configuration System

1. Add schema-level config validation at startup.
2. Add explicit config versioning and migration path.
3. Define environment and secret handling policy.

Factory and Templates

1. Build typed template registry and dispatch.
2. Separate template metadata from adapter execution logic.
3. Preserve template capability toggles (audit, journal, locking, cache, retention).

Observability and Lineage

1. Keep and formalize event lineage fields.
2. Add structured telemetry for queue lag, throughput, retries, and latency distributions.
3. Provide operator-facing diagnostics for stuck consumers and poison messages.

Security Baseline

1. Enforce authenticated producer and consumer connectivity.
2. Define service-origin trust checks for internal events.
3. Centralize authorization for template access boundaries.

Data and Cache Behavior

1. Define cache invalidation contracts for write events.
2. Define read consistency options per template.
3. Add optimistic concurrency where needed.

Verification and Testing

1. Add message-contract compatibility tests.
2. Keep integration tests with real RabbitMQ in CI where possible.
3. Add failure-path and soak test milestones before code-complete.

Execution Sequence

Phase A: POC Transport Stability (Now)

Goal:

• Prove broker startup and message flow for rec.read and rec.write.

Definition of done:

• Spawn tests pass.
• Message round-trip smoke tests pass.
• Wiki reflects current tested behavior.

Phase B: Minimal Core Dispatch

Goal:

• Replace handler stubs with basic factory wiring for one template path.

Definition of done:

• fetch and write events route through a real dispatch path.
• End-to-end request and response schema is stable for first template.

Phase C: Guardrail Install

Goal:

• Add reliability and validation layers without breaking proven POC behavior.

Definition of done:

• Deterministic ingress validation.
• Explicit ack/nack behavior.
• Basic dead-letter and retry policy in place.

Phase D: Productization Baseline

Goal:

• Hardening for repeatable deployment and operations.

Definition of done:

• Supervision and graceful shutdown.
• Operator diagnostics and telemetry.
• Config schema and migration strategy.

Phase E: White-Paper Readiness

Goal:

• Demonstrate zero core code changes for new product behavior.

Definition of done:

• SPEW backend behavior enabled through config and template/domain declarations.
• Benchmark and lineage evidence captured for white-paper material.

Acceptance Gates

1. Functionality gate: broker flow and dispatch correctness.
2. Reliability gate: bounded behavior under failures.
3. Operability gate: observable and diagnosable runtime.
4. Product gate: new domain behavior without core edits.

Documentation Rule

Any substantive architecture or runtime behavior change must update relevant wiki pages in the same change set.