milestone: enforce required services and add mongo logger PoC path

This commit is contained in:
2026-04-06 18:16:48 -07:00
parent dd04fb5168
commit 836a968806
17 changed files with 1081 additions and 112 deletions

View File

@@ -12,9 +12,10 @@ The IPL sequence is not arbitrary. Each step depends on the previous one:
1. **Configuration must load first** — every subsequent step reads from it
2. **Logging must initialize second** — every subsequent step may emit log events
3. **RabbitMQ must be reachable third** — it is the transport for everything, including log event routing to admin
4. **MongoDB must be reachable fourth** — it is the log persistence store on the admin node, and the primary document store on appServer
5. **MariaDB must be reachable fifth** — it is the relational store; non-critical in dev but required in production
3. **Runtime templates must validate third** — startup must reject invalid template manifests before service checks
4. **RabbitMQ must be reachable fourth** — it is the transport for everything, including log event routing to admin
5. **MongoDB must be reachable fifth** — it is the log persistence store on the admin node, and the primary document store on appServer
6. **MariaDB must be reachable sixth** — it is the relational store; non-critical in dev but required in production
You cannot initialize logging before loading config because the log destination (syslog vs console, mirror settings) is in the config. You cannot validate RabbitMQ before initializing logging because you need logging to report the result. The order is a dependency chain, not a preference.
@@ -40,9 +41,28 @@ Initializes the `tracing` subscriber with journald and/or console output based o
**Why second:** Config is loaded. Logging destination is known. Every step from here on can emit structured log output.
**Note on log routing:** At this point, log output goes to the local console and/or journald. Log events are not yet routed to the admin node's MongoDB `msLogs` collection — that requires RabbitMQ to be up (Step 3). Local logging is the fallback that covers the gap between process start and AMQP connectivity.
**Note on log routing:** At this point, log output goes to the local console and/or journald. Log events are not yet routed to the admin node's MongoDB `msLogs` collection — that requires RabbitMQ to be up (Step 4). Local logging is the fallback that covers the gap between process start and AMQP connectivity.
### Step 3: Validate RabbitMQ Reachability (TCP)
### Step 2b: Load and Validate REC Templates
```rust
let runtime_templates = template_registry::load_runtime_rec_templates("templates")
```
Loads every TOML template under `templates/`, selects `schema = "rec"`, and validates internal consistency before network service validation begins.
Current validation scope includes:
- Every protected field must exist in `fields`.
- Every declared index/cache/regex/partial index field must exist in `fields`.
- Every `compound_indexes` name must appear in `index_name_list`.
**Why here:** this catches template drift and invalid declarations early, before brokers or adapters process traffic.
**Environment-aware failure handling:**
- `production`: invalid template registry is fatal
- all other environments: invalid registry is warning-only so local POC workflows can continue
### Step 4: Validate RabbitMQ Reachability (TCP)
```rust
match services::amqp::validate(&cfg.broker_services) { ... }
@@ -56,7 +76,7 @@ Opens a TCP connection to the configured RabbitMQ broker host and port. Does not
- `production`: unreachable broker is fatal — the node cannot function
- all other environments: unreachable broker is a warning — IPL continues so developers can work on other components without a running broker
### Step 3b: Authenticate to RabbitMQ + Declare Exchange
### Step 4b: Authenticate to RabbitMQ + Declare Exchange
```rust
let amqp_conn = match services::amqp::AmqpConnection::connect(&cfg.broker_services).await { ... }
@@ -74,7 +94,7 @@ Opens a full AMQP session — credentials, vhost, and channel. Then asserts the
- `production`: authentication failure is fatal
- all other environments: failure is a warning — `amqp_conn` is `None`, IPL continues
### Step 4: Validate MongoDB
### Step 5: Validate MongoDB
```rust
match mongo::validate_all(&cfg.rec_services) { ... }
@@ -86,7 +106,7 @@ Opens a TCP connection to each configured MongoDB node. One entry per BEDS servi
**Environment-aware failure handling:** Same pattern as RabbitMQ — fatal in production, warning in development.
### Step 5: Validate MariaDB
### Step 6: Validate MariaDB
```rust
match mariadb::validate_all(&cfg.rel_services) { ... }

View File

@@ -159,6 +159,13 @@ Current proof-of-concept verification for the two active appServer brokers is co
- `tests/broker_pool_test.rs` validates that configured rBroker/wBroker pool instances spawn.
- `tests/broker_message_flow_test.rs` validates end-to-end message flow by publishing `ping` events to
`rec.read` and `rec.write` and asserting broker replies.
- `tests/broker_message_flow_test.rs` also validates logger sequence round-trip by publishing a `write`
event to `rec.write` with `template="Logger"`, then fetching via `fetch` on `rec.read` and asserting
that the newly written log message is returned.
Current logger sequence in POC now writes and reads through MongoDB (`msLogs`) via the
`Logger` template path (`rec.write` for write, `rec.read` for fetch), using credentials from
the active environment config.
These tests provide a lightweight deployment confidence check while the framework is still in the
"POC before guardrails" phase.

View File

@@ -12,6 +12,21 @@ templates/
└── mst_logger_rec.toml ← logger collection template
```
## Runtime Template Registry (Implemented)
At IPL, BEDS now loads REC templates from `templates/` into a typed runtime registry and validates them before service connectivity checks.
Current runtime validation enforces:
- Any field listed in `protected_fields` must exist in `[fields]`.
- Any field listed in `index_fields` must exist in `[fields]`.
- Any key in `[single_field_indexes]`, `[unique_indexes]`, and `[ttl_indexes]` must exist in `[fields]`.
- Every field used in `[compound_indexes]` and `partial_indexes` qualifiers must exist in `[fields]`.
- Every field used in `regex_pattern_indexing` and `[cache_map]` must exist in `[fields]`.
- Every `compound_indexes` index name must be present in `index_name_list`.
In `production`, template registry validation failures are fatal during IPL. In non-production environments, failures are warning-only to preserve local POC workflows.
## Two Template Types
| Type | Schema | Database | Use Case |

View File

@@ -48,6 +48,10 @@ Immediate objective:
Guardrails are intentionally deferred until POC behavior is stable.
Implementation status update:
- Phase A transport stability evidence exists: live RabbitMQ round-trip tests for `rec.read` and `rec.write` ping paths.
- Phase B has started: REC template registry loading and startup validation are now implemented in IPL.
## Must-Keep Invariants
1. AMQP-first data path for application operations.