Polyglot Part 2: BEJSON 105 Core & MFDB v1.40 Across Python, JS, TS, and Bash

Chapter 2: Format 105 Structural DNA — The Three Flavors (105, 105a, 105db) & Vector Offsets

If you don't understand the physical byte layout of a format, you're not an engineer — you're just guessing. Format 105 comes in three distinct flavors, each tailored for a precise persistence boundary:

The Reserved record_uuid Vector Offset Law

Look closely at how rows are constructed. In standard 104, if you declare 3 fields, your row vector has 3 elements. In 105, every single row vector is prepended with an immutable record_uuid:

// Format 105 Dense Table Anatomy
{
  "Format": "BEJSON",
  "Format_Version": "105",
  "Format_Creator": "Elton Boehnen",
  "Records_Type": ["Author"],
  "Fields": [
    {"name": "author_code", "type": "string", "field_uuid": "00000000-0000-4000-8000-000000000001"},
    {"name": "author_name", "type": "string", "field_uuid": "00000000-0000-4000-8000-000000000002"}
  ],
  "Values": [
    ["11111111-1111-4111-8111-111111111111", "auth_01", "Ada Lovelace"],
    ["22222222-2222-4222-8222-222222222222", "auth_02", "Alan Turing"]
  ]
}

Notice that Fields has 2 field definitions, but every row in Values has 3 cells! Column 0 is the row's physical record_uuid. Therefore, the absolute Values-row column for field j is mathematically defined as:

def bejson_validator_row_col(version, field_idx):
    if version == "105db":
        return 0 if field_idx == 0 else field_idx + 1
    return field_idx + (1 if version in ("105", "105a") else 0)

In 105db, the discriminator (Record_Type_Parent) sits at column 0, while record_uuid immediately follows at column 1, and the payload fields follow at column 2 onwards.

The Fail-on-Switch Invariant

Corporate developers love silent fallbacks. If someone bumps a version string from 104 to 105 without compiling UUIDs, bloated frameworks will "auto-guess" or silently pretend everything is fine. In BEJSON, that behavior is considered a critical defect. The Fail-on-Switch invariant guarantees that if Format_Version claims 105-series, but any field lacks a field_uuid or any row lacks a record_uuid, the engine immediately halts with E_INVALID_FIELDS (Code 6) or E_INVALID_VALUES (Code 7). Zero silent drift. Zero data corruption.

Chapter 3: Subsystem A (bejson_core) — The Provenance-Guarded Dual-Key Field Cache

Every noob writes code with hardcoded numbers: row[1] or row[2]. When you add a column or swap fields, your entire script crashes or, even worse, corrupts your data silently. In Policy §6.2 and §3.2, hardcoded indices are strictly banned. But in Format 105, because every row has a reserved record_uuid offset, and every field has a unique field_uuid, doing manual offset math (idx + 1) in application code is an invitation to disaster.

To eliminate this, Subsystem A in libraries_105/Lib_PY/Core/lib_bejson_Core_bejson_core.py (and its JS/TS/SH siblings) introduces the Multi-Index Field Cache:

# Multi-Index Field Cache Topology in BEJSON 105
{
  "by_name":     {"author_code": 0, "author_name": 1},
  "by_uuid":     {"00000000-0000-4000-8000-000000000001": 0, "00000000-0000-4000-8000-000000000002": 1},
  "by_name_row": {"author_code": 1, "author_name": 2}
}

The by_name_row Abstraction

When you call bejson_core_get_field_map(doc) on a Format 105 document, the engine doesn't return the raw Fields array indices. It returns by_name_row! Look at what this means for the application developer:

# The caller NEVER has to do offset arithmetic!
field_map = bejson_core_105_get_field_map(doc)

for row in doc["Values"]:
    # field_map["author_name"] automatically evaluates to index 2!
    author = row[field_map["author_name"]]
    code = row[field_map["author_code"]]

Zero offset math. Constant-time $O(1)$ lookups. Exact column resolution. And if you need to resolve a field by its immutable UUID across distributed migrations, by_uuid gives you instantaneous resolution.

Dual-Key Integrity & Error Code 80

What happens if a rogue developer tries to query a field by name and by UUID, but the two definitions point to different schema slots? Subsystem A enforces Dual-Key Cross-Verification. If a name-to-slot mapping disagrees with its UUID-to-slot mapping, the engine halts with E_FIELD_INTEGRITY_MISMATCH (Error Code 80). This guarantees that schema renames never corrupt the relational pipeline.

The Cache Invalidation Protocol (_bejson_core_invalidate_cache)

In-memory caches can become stale if fields are dynamically renamed or mutated. In clarification.md §5.6, the core engine specifies a strict invalidation protocol: calling _bejson_core_invalidate_cache(doc) wipes the in-memory cache and forces a rebuild upon next access. Furthermore, per Policy §5.2 (`E10`), in-memory cache keys starting with _ (like _bejson_field_map) are strictly forbidden from ever being persisted to disk. If a file on disk contains a top-level key prefixed with _, the validator instantly rejects it with E_RESERVED_KEY_COLLISION (Error Code 10).

Chapter 4: Subsystem B (bejson_validator) — Strict Integrity Gates, Fail-on-Switch & Error Codes 80–85

Most validation engines in web development are an absolute joke. They accept arbitrary keys, perform sloppy type coercion (converting "123" to 123 behind your back), and dump 500 lines of unreadable stack traces. In Subsystem B (lib_bejson_Core_bejson_validator), validation is a deterministic, multi-stage pipeline where every error is mapped to an immutable numerical code.

The 105 Strict Integrity Gates

When the validator detects a 105-series document, it activates bejson_validator_check_105_strict_integrity(). This stage executes three uncompromising verification gates:

1. Field UUID Gate: Every element in Fields must contain a valid field_uuid string. If missing, it raises E_INVALID_FIELDS (Code 6). If two fields share the same UUID, it halts with E_DUPLICATE_FIELD_UUID (Code 85). If two fields share the same name, it halts with E_DUPLICATE_FIELD_NAME (Code 84).
2. Row Vector Cardinality & Record UUID Gate: Every row in Values must have exactly len(Fields) + 1 cells (accounting for the record_uuid slot). If a row has too few or too many cells, it halts with E_RECORD_LENGTH_MISMATCH (Code 9). Furthermore, every row's record_uuid must be non-null, unique across the entire document, and located at the exact expected offset (col 0 for 105/105a, col 1 for 105db). A duplicate record UUID raises E_INVALID_VALUES (Code 7).
3. Fail-on-Switch Gate: If a document has "Format_Version": "105" but was never processed through the upgrade pipeline (e.g. Someone manually edited the version header), the validator catches this immediately and outputs a clear diagnostic instruction: "Format_Version claims 105-series but Fields were never compiled with field_uuid keys -- run bejson_core_upgrade_to_105() before using this file."

Advisory UUIDv4 Format Checking

Does the engine choke if you use an RFC 4122 v5 UUID or an enterprise UUID with uppercase letters? No. The function bejson_validator_check_105_uuid_format() runs advisory regex validation (^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$). If non-conforming UUIDs are present, they are reported as warnings in the diagnostic report rather than fatal errors. This gives developers complete strictness for runtime operations without breaking interoperability with external systems.

The 105-Series Error Code Registry

Here is the definitive numerical error map for Format 105 operations across all four runtimes:

Chapter 5: Subsystem C (mfdb_core v1.40) — Multi-File Database Relational Backbone & Ref_Integrity

Every junior developer thinks "relational" means you have to run an SQL daemon, configure connection pools, and manage bloated schema migration scripts. That is pure corporate brain rot. Elton Boehnen proved that you can have strict, enterprise-grade relational integrity over simple, human-readable POSIX flat files with MFDB v1.40.

The Ref_Integrity Manifest Switch

In MFDB v1.31, manifests federated files and verified row counts. In MFDB v1.40 (backed by lib_bejson_Core_mfdb_core), the database manifest introduces the Ref_Integrity flag:

// 105a.mfdb.bejson Manifest Header
{
  "Format": "BEJSON",
  "Format_Version": "105a",
  "Format_Creator": "Elton Boehnen",
  "Records_Type": ["mfdb"],
  "Network_Role": "Master",
  "Ref_Integrity": true,
  "Fields": [
    {"name": "entity_name", "type": "string"},
    {"name": "file_path", "type": "string"},
    {"name": "record_count", "type": "integer"}
  ],
  "Values": [
    ["Authors", "data/authors.105.bejson", 42],
    ["Books", "data/books.105.bejson", 128]
  ]
}

Subsystem C provides dedicated control primitives to toggle and inspect this state across all languages:

• mfdb_core_enable_ref_integrity(manifest_path): Atomically flips Ref_Integrity: true in the manifest.
• mfdb_core_disable_ref_integrity(manifest_path): Atomically flips Ref_Integrity: false (useful for bulk batch imports).
• mfdb_core_is_ref_integrity_enabled(manifest_path): Inspects the manifest and returns a boolean.

Writing with Integrity: mfdb_core_add_entity_record_105

When you insert a row into a 105 entity table, you do not use the legacy 104 function (which would raise E_FORMAT_UNSUPPORTED [81]). You call mfdb_core_add_entity_record_105(). Look at what happens under the hood during this single call:

# Inserting a record into an MFDB v1.40 database
result = mfdb_core_add_entity_record_105(
    manifest_path="105a.mfdb.bejson",
    entity_name="Books",
    record_values=["The C Programming Language", author_uuid]
)
# result -> {"record_uuid": "e3b0c442-98fc-1c14-9afb-4c72e04f0042"}

1. The engine mints a fresh, guaranteed-unique UUIDv4 for the record.
2. If Ref_Integrity is true, the engine scans the schema for foreign key columns (e.g. author_id with fk_target_entity: "Authors").
3. It looks up the target table (data/authors.105.bejson) and verifies whether author_uuid exists.
4. The Dangling Pointer Trap: If author_uuid is invalid or missing, the operation halts immediately with E_MFDB_FK_UNRESOLVED (Error Code 39)! No dirty data is written to disk. The transaction is aborted.
5. If the reference is valid (or null), the row is committed using two-phase atomic replacement (.tmp → fsync → os.replace).
6. The manifest's record_count column is automatically updated and synced in the same atomic cycle.

Chapter 6: Subsystem D (mfdb_validator Phase 6) — Active Referential Constraints, Cascades & Cycle Defense

Every database looks great on day one when you only insert clean test rows. Real production software gets tested when things get deleted. When you delete a parent record in an interconnected schema, does your database leave broken dangling pointers, lock up in infinite loops, or wipe out the wrong tables? In MFDB v1.40 Phase 6, Subsystem D (lib_bejson_Core_mfdb_validator) provides mathematically verified referential integrity.

The Four Deletion Policies (fk_on_delete)

In Format 105 schema definitions, foreign key fields declare their relationship target (fk_target_entity) and their deletion behavior (fk_on_delete):

Subsystem D Cycle Defense: Halting Infinite Cascade Loops

Here is an edge case that destroys amateur database engines: Table A has an FK pointing to Table B with cascade, and Table B has an FK pointing to Table A with cascade. If you delete a row in Table A, a naive cascade engine will enter an infinite recursion loop, consuming 100% of the CPU and crashing with a stack overflow.

Subsystem D implements a Directed Acyclic Deletion Graph (DADG) tracker. Before any file on disk is touched, the cascade resolver traverses the foreign key graph. If a circular reference loop is detected in the cascade chain, the operation halts immediately with E_MFDB_FK_CASCADE_CYCLE (Error Code 45). Zero stack overflows. Zero infinite loops.

Standalone Referential Integrity Auditing

You don't have to wait for a runtime delete to verify your database's health. You can execute a zero-mutation, read-only audit at any time across Python, JS, TS, and Bash:

# Python audit invocation
from lib_bejson_Core_mfdb_validator import mfdb_validator_check_referential_integrity

report = mfdb_validator_check_referential_integrity("105a.mfdb.bejson")
if not report["is_valid"]:
    for err in report["errors"]:
        print(f"DEFECT [{err['code']}]: {err['message']}")

The auditor scans every registered entity, checks every FK against its target PK table, identifies unresolved references ([39]), catches missing parent manifests ([36], [37]), and detects unregistered entities claiming parentage ([38]).

Chapter 7: Subsystem E — The Atomic 104-to-105 Migration Pipeline & Verbatim Backups

How do real software engineers upgrade live production databases? They don't run interactive wizard GUIs or pray during an in-place modification. They use Subsystem E (The Atomic 104-to-105 Migration Engine) in Lib_PY/Core/lib_bejson_Core_bejson_upgrade.py and its dedicated CLI tools.

The 7-Step Zero-Risk Upgrade Protocol

When you run bejson_upgrade_cli.py --input "data/legacy.bejson" --atomic, the engine executes a fail-safe, transactional sequence:

1. Pre-Flight Baseline Validation: The legacy file is parsed and validated against the strict 104 specification. If the file is already corrupted or malformed, the pipeline halts immediately without touching disk.
2. Verbatim Safety Snapshot: The file is copied bit-for-bit to a dedicated backup directory: Backups/<filename>.104.bak.
3. Field UUID Synthesis: The engine iterates through the Fields array. For every field dictionary, it mints a fresh, deterministic RFC 4122 UUIDv4 and injects it as field_uuid while preserving field ordering.
4. Record UUID Prepending: The engine iterates through Values. For every row vector, it mints an immutable record_uuid and prepends it at the format-correct column (col 0 for 104/104a → 105/105a; col 1 for 104db → 105db).
5. Header Transformation: The version header is bumped: "104" → "105", "104a" → "105a", or "104db" → "105db".
6. Post-Upgrade Stage Gate: Before committing, the generated document is evaluated against bejson_validator_check_105_strict_integrity(). If any verification defect surfaces, the operation halts with E_UPGRADE_VALIDATION_FAILED (Error Code 83), the staged file is discarded, and the original file is left pristine.
7. POSIX Two-Phase Atomic Swap: The validated data is flushed to a temporary file (.tmp), hardware-synced with fsync(), and atomically renamed onto the live target using os.replace() / fs.renameSync().

Whole-Database Migration: upgrade_mfdb_to_105.py

What if you need to upgrade an entire Multi-File Database containing 50 interconnected entity tables? You don't upgrade files manually like a peasant. You invoke the whole-database coordinator:

# Upgrades manifest, all child entity tables, and verifies relational integrity
python3 Lib_PY/Core/cli/upgrade_mfdb_to_105.py --manifest "104a.mfdb.bejson"

The coordinator discovers all registered entity tables, backs up the entire manifest tree, upgrades each table atomically, updates the manifest to 105a.mfdb.bejson, enables Ref_Integrity: true, and executes a full Phase 6 referential integrity audit. If even a single entity fails, the entire database rollbacks cleanly to the pre-upgrade snapshot.

Chapter 8: Subsystems F & G — The 4-Language Polyglot Parity Matrix & AI Vector Economics

You’ve seen the physical format layout, the dual-key cache, the strict validation gates, the MFDB v1.40 relational engine, and the atomic upgrade pipeline. Now let’s lock down the complete 4-Language Polyglot Parity Reference and dissect the AI Vector Economics that make BEJSON 105 the ultimate data architecture for distributed AI agents.

The Universal 4-Language Integrity Era Parity Matrix

Every single 105 operation is matched 1:1 across Python, JavaScript, TypeScript, and POSIX Bash:

Unified Error Code Cross-Registry

In fragmented platforms, an error code in Python means something completely different in JavaScript or Bash. In the BEJSON ecosystem, error codes are globally unified constants:

• Codes [43]–[47] (Referential Integrity Phase 6): Target entity unknown (43), restrict violation (44), cascade cycle loop (45), entity still referenced (46), invalid on_delete policy (47).
• Codes [80]–[85] (Format 105 & Upgrade): Dual-key mismatch (80), format unsupported (81), record not found (82), upgrade validation failed (83), duplicate field name (84), duplicate field UUID (85).

AI Vector Economics: Why Relational Agents Run on BEJSON 105

When autonomous AI agents interact with relational datasets, standard JSON structures are a performance nightmare. Let's look at the actual token telemetry when an AI agent needs to parse a relational catalog:

Why is the reduction so massive? Because standard JSON forces the LLM to process repetitive keys ("author_uuid": "...", "book_uuid": "...") thousands of times across the context window. In BEJSON 105, the schema and field UUIDs are declared once at the top, allowing prompt caching engines (like Gemini's context cache) to freeze the schema header in GPU memory. The model only reads and writes pure positional arrays. No hallucinated property names. No missing keys. 100% deterministic accuracy.