Polyglot Part 1: BEJSON 104 Core Across Python, JS, TS, and Bash

Chapter 1: The Polyglot Parity Architecture — 4 Runtimes, 1 Immutable Spec

Ayo noobs, look at the disaster that is modern enterprise software. Company A writes their backend ingestion in Python, their microservices in Node.js, their frontend in TypeScript, and their infrastructure deployment in Bash. What happens? Python dumps an object, JavaScript parses it with dynamic key coercion, TypeScript complains about untyped properties, and the Bash script mangles quotes with sed and awk until production crashes at 3:00 AM. Total amateur hour.

If your serialization format doesn't have 100% strict cross-language behavioral parity, your distributed architecture is built on sand. That is the exact engineering challenge Elton Boehnen solved with the BEJSON 104 Core Polyglot Parity Family (libraries_104).

The Four Runtimes of the Apocalypse

The Core family is not a set of loose translations. It is an identical, synchronized engine implemented natively in four primary language environments:

1. Python (Lib_PY/Core): The reference engine. 100% zero-dependency Python standard library (json, os, sys, pathlib, hashlib, typing). Pure POSIX compliance, executed across standard Linux and Android Termux ARM64.
2. JavaScript (Lib_JS/Core): CommonJS / Node.js 18+ runtime. Zero npm dependencies. Implements identical memory caching, validation algorithms, and atomic write pipelines.
3. TypeScript (Lib_TS/Core): Type-safe implementation featuring strict interfaces (BEJSONDocument, FieldDefinition, ValidationReport). Provides compile-time contract enforcement while maintaining identical runtime behavior.
4. Bash / Shell (Lib_SH/Core): The sysadmin's dream. Pure POSIX shell functions leveraging standard core utilities (jq, sed, grep, awk). Allows low-level daemon automation without requiring Python or Node runtimes!

The Zero-Dependency POSIX Law

Notice what all four runtimes share: ZERO EXTERNAL DEPENDENCIES. There are no pip install requirements, no node_modules folders with 40,000 untrusted third-party dependencies, and no compiled C-libraries. You can clone libraries_104 onto an air-gapped server, a minimal Docker container, or an Android phone in Termux, and the entire database engine bootstraps in under 5 milliseconds. That's real engineering.

Chapter 2: Subsystem A (bejson_core) — Memory I/O, Atomic Swaps & Vector Mutations

Every database engine lives or dies by its low-level memory primitives. If your I/O layer corrupts memory buffers or writes partial files to disk during an unexpected reboot, everything upstream dies. In the 104 Core family, all low-level memory allocation, file parsing, vector manipulation, and atomic commits are governed by Subsystem A: bejson_core.

Core Functional Primitives

Whether you are calling Python (lib_bejson_Core_bejson_core.py), JavaScript (lib_bejson_Core_bejson_core.js), TypeScript (lib_bejson_Core_bejson_core.ts), or Bash (lib_bejson_Core_bejson_core.sh), you have access to the exact same operational verbs:

• bejson_core_load_file(path): Reads UTF-8 bytes from disk, parses JSON syntax, audits root dictionary bounds, and primes the in-memory field-map cache.
• bejson_core_parse_bejson(str): Parses an in-memory string payload into a live BEJSON document structure.
• bejson_core_insert_record(doc, values): Validates that len(values) == len(Fields) before mutating memory. Appends the row vector to Values. Returns the new total row count.
• bejson_core_update_record(doc, row_idx, values): Verifies zero-based bounds (bash \le ext{row\_idx} < ext{len(Values)}$), audits vector length parity, and updates the row in-place.
• bejson_core_delete_record(doc, row_idx): Safely removes the target row vector from the Values matrix.
• bejson_core_find_records(doc, field_name, value): Fast query routine utilizing the (1)$ field-map cache to return all matching (row_index, row_values) tuples.

The Two-Phase Atomic Persistence Protocol

Skids use open(path, 'w').write(). Real hackers know that writing directly to an active file on disk is an invitation for disaster. If the process is killed mid-write, or Termux runs out of battery, your database file is truncated to 0 bytes and destroyed forever.

Every core runtime strictly enforces Two-Phase POSIX Atomic Persistence:

[In-Memory Document]
         │
         ▾
1. Strip runtime memory keys ('_bejson_field_map')
         │
         ▾
2. Write serialized JSON to staging file (<target>.tmp)
         │
         ▾
3. Flush OS Kernel Buffers (fsync)
         │
         ▾
4. Atomic POSIX Filesystem Replace (os.replace / rename)
         │
         ▾
[Final Target File on Disk] (Zero partial write risk!)

Because POSIX rename() is atomic at the filesystem inode level, the switch happens instantaneously. Readers either see the complete previous version or the complete new version. Zero partial writes, zero corrupt states.

Chapter 3: Subsystem B — The O(1) Field-Map Cache & Boundary Sanitization Engine

Here is where amateur coders write software that chokes on large datasets. Suppose you have a table with 50,000 rows and 20 columns. An amateur writes a helper function that loops through Fields to find the index of "ip_address" on every single row iteration. That’s 1,000,000 string comparisons in Python or JavaScript just to read one column! Your CPU cache lines are constantly thrashing. Lmao.

In the 104 Core family, this anti-pattern is eradicated at the engine level by Subsystem B: The O(1) Field-Map Cache.

How the Field-Map Cache Works

def bejson_core_get_field_map(document: dict) -> dict:
    # 1. Check if cached map is already attached in memory:
    if "_bejson_field_map" in document:
        return document["_bejson_field_map"]
        
    # 2. Compile hash map: field_name -> column index
    fields = document.get("Fields", [])
    field_map = {field_def["name"]: idx for idx, field_def in enumerate(fields)}
    
    # 3. Cache directly on document object for subsequent lookups
    document["_bejson_field_map"] = field_map
    return field_map

When you call bejson_core_get_field_map(), the runtime compiles the field map once in (K)$ time and caches it in memory under the private key _bejson_field_map. Subsequent lookups in tight loops execute in guaranteed (1)$ constant time.

Automatic Invalidation via FNV-1a Hashing

What happens if your code adds or reorders columns in memory? If you kept reading with a stale cache, you'd corrupt data. The Core engine computes a 64-bit FNV-1a hash over the Fields schema structure. If the schema fingerprint mutates, the runtime automatically invalidates the old map and recompiles a fresh cache on the next access. Zero manual cache management required.

Boundary Sanitization Protocol (Policy §10.9)

Raw data ingested from CSVs, external APIs, or user input is often dirty. The Core engine provides bejson_core_sanitize_data() as an automated sanitization gate:

1. Key Sanitization: Strips trailing whitespace, non-printable ASCII bytes, and null characters from field names.
2. Value Sanitization: Trims boundary whitespace from string cells unless explicitly preserved as raw code blocks.
3. Ragged Row Normalization: Incomplete rows missing trailing columns are automatically padded with declared schema defaults or null, preventing array out-of-bounds crashes.

Chapter 4: Subsystem C (bejson_validator) — 20-Point Deterministic Stage Gates & Error Registry

You know what makes most database drivers unreliable? Vague error messages. You get a generic "SyntaxError: Unexpected token at row 42" and spend 3 hours guessing what field failed. In the BEJSON 104 Core family, validation is not an afterthought—it is a deterministic 20-Point Stage-Gate Pipeline governed by Subsystem C: bejson_validator.

The 5 Validation Stages

Master Core Error Code Registry (Codes 1–29)

Every failure maps to an exact integer code and symbolic token across all four language implementations:

• Code 1 (E_FILE_NOT_FOUND): Target file does not exist on disk or path traversal attempted.
• Code 2 (E_JSON_PARSE_FAILED): Malformed JSON syntax in source file.
• Code 3 / 4 (E_MISSING_FORMAT / E_INVALID_FORMAT): Missing or invalid "Format" key (must equal "BEJSON").
• Code 5 / 6 (E_MISSING_VERSION / E_INVALID_VERSION): Version key missing or not in approved format list.
• Code 7 / 8 (E_MISSING_CREATOR / E_INVALID_CREATOR): Creator header missing or does not equal "Elton Boehnen".
• Code 9 (E_MISSING_RECORDS_TYPE): Entity definition array missing or empty.
• Code 12 (E_DUPLICATE_FIELD): Duplicate column name declared in Fields.
• Code 16 (E_ROW_LENGTH_MISMATCH): Ragged row detected! Row element count deviates from len(Fields).
• Code 17 (E_CELL_TYPE_MISMATCH): Cell value violates declared schema type (e.g. text passed in an integer column).
• Code 19 (E_ATOMIC_WRITE_FAILED): Disk full or permission denied during atomic staging swap.

Chapter 5: Subsystem D (mfdb_core) — Multi-File Database Federation Engine

When you have 50 different data models (users, sessions, payments, articles, categories), stuffing them into one giant file is a terrible idea. In the 104 Core family, multi-file domain partitioning is coordinated by Subsystem D: mfdb_core and audited by mfdb_validator.

The Architecture of an MFDB (v1.31 / v1.39)

An MFDB on disk is structured as an authoritative root manifest coordinating dense entity files:

site_master/
  104a.mfdb.bejson         ← Central Manifest (Format 104a)
  data/
    category.bejson        ← Entity File (Format 104, Dense)
    pagerecord.bejson      ← Entity File (Format 104, Dense)
    authorprofile.bejson   ← Entity File (Format 104, Dense)

• Manifest Specification: Always named 104a.mfdb.bejson. Format_Version is strictly "104a", and Records_Type is strictly ["mfdb"]. Mandatory headers include MFDB_Version and DB_Name.
• Dense Entity Files: Each entity file stores exactly one domain model. Unlike 104db, there is zero null-padding across entities. Data is dense, lean, and cache-friendly.

Bidirectional Hierarchy Resolution (The Anti-Orphan Invariant)

In traditional databases, deleting a table from your schema leaves stray data files rotting on disk. In MFDB, orphan files are physically impossible due to Bidirectional Hierarchy Resolution:

1. The manifest declares the entity with a relative path: ["Pages", "data/pages.bejson"].
2. The entity file in data/pages.bejson MUST contain a Parent_Hierarchy header pointing directly back to the manifest: "Parent_Hierarchy": "../104a.mfdb.bejson".
3. The validator checks both directions. If the manifest lists a file that doesn't point back, OR if an entity file points to a manifest that doesn't list it, validation fails immediately with Code 38 (Bidirectional Path Mismatch) or Code 36 (Parent Hierarchy Unresolved)!

Master MFDB Error Codes (Codes 30–60)

• Code 30 (E_MFDB_MANIFEST_NOT_FOUND): Root 104a.mfdb.bejson is missing.
• Code 31 (E_MFDB_MANIFEST_INVALID): Manifest fails 104a schema validation.
• Code 32 / 33 (E_MFDB_ENTITY_NOT_FOUND / E_MFDB_ENTITY_INVALID): Entity file missing on disk or fails 104 validation.
• Code 34 (E_MFDB_DUPLICATE_ENTITY): Duplicate entity name declared in manifest.
• Code 35 (E_MFDB_INVALID_ROLE): Network_Role header must strictly be "Master", "Slave", or "Standalone".
• Code 36 (E_MFDB_PARENT_HIERARCHY_UNRESOLVED): Entity's Parent_Hierarchy relative path is broken.
• Code 37 (E_MFDB_CIRCULAR_HIERARCHY): Circular parent-child containment detected across databases.

Chapter 6: Subsystem E (bejson_core_nesting) — In-Cell Hierarchical Nesting & Dotted Queries

Complex systems don't fit into flat 2D grids. When you model security access control lists (ACLs), container process trees, or compiler abstract syntax trees (ASTs), data is inherently hierarchical. In MongoDB or raw JSON, developers just dump arbitrary nested objects with zero schema contracts, creating an untyped nightmare.

In the 104 Core family, hierarchical nesting is solved mathematically by Subsystem E: bejson_core_nesting.

The NestAddress Coordinate Standard

Every nested child document embedded inside a parent cell is referenced by a deterministic 4-dimensional coordinate tuple:

NestAddress = (parent_file_path, row_index, col_index, depth)

Example: ("/storage/logs.bejson", 12, 4, 1) uniquely pinpoints the child BEJSON table embedded at row 12, column 4 of logs.bejson at recursion depth 1. No synthetic foreign key columns required!

The Uniform Column-Schema Gate (The Anti-Chaos Rule)

If you designate column 4 to hold nested process tables, the engine enforces Uniform Column-Schema Contracts:

• Every child document in that column must possess an identical Fields array with matching names, types, and sequence order.
• If row 0 stores child records with ["pid", "comm"], row 1 cannot embed ["name", "status"]. Violating this triggers Code 131 (E_NESTING_SCHEMA_MISMATCH)!
• The 16-Depth Hard Ceiling: To prevent stack overflows or recursive zip-bomb attacks, recursion is capped at 16 levels (NESTING_MAX_DEPTH). Exceeding depth 16 triggers Code 132 (E_NESTING_MAX_DEPTH_EXCEEDED).

Dotted-Path Navigation Engine

You traverse multi-level nested tables using the bejson_nesting_query() engine, implemented with identical syntax in Python, JS, TS, and Bash:

# Extracts all target process names from nested tables across all hosts:
running_pids = bejson_nesting_query(cluster_doc, "host_processes[*].pid")
# Returns: [1, 284, 512, 1337]

Nesting Error Registry (Codes 130–159)

• Code 130 (E_NESTING_INVALID_CHILD): Cell payload is not a valid BEJSON child document.
• Code 131 (E_NESTING_SCHEMA_MISMATCH): Child document schema deviates from the column contract.
• Code 132 (E_NESTING_MAX_DEPTH_EXCEEDED): Nesting exceeds safety limit of 16 levels.
• Code 133 (E_NESTING_CIRCULAR_REFERENCE): Circular reference detected (child references parent).
• Code 134 (E_NESTING_ADDRESS_INVALID): NestAddress coordinate out of bounds.

Chapter 7: Subsystems F & G — Chunking Archives, PathGuard Defense & Transaction Journals

A true production database ecosystem doesn't stop at table reads and writes. You need tools to transport entire repositories, protect against directory traversal security exploits, and manage transaction rollbacks when an autonomous agent makes a mistake. The 104 Core family packages these into specialized submodules.

Subsystem F: Single-File Container Chunking (bejson_chunking)

Ever tried to send a 2,000-file codebase to an AI model or across an air-gapped network? Zip files are binary blobs that LLMs can't read natively. bejson_chunking collapses entire directory trees, MFDB databases, and repositories into a single portable text file (Format 104a):

• The Chunked-104a Schema: The container uses a strict schema with columns: file_path, file_size_bytes, file_sha256, file_encoding ("utf-8" or "base64"), and file_content.
• Virtual Mounting: The runtime mounts the chunked archive in memory. Applications read files virtually without having to extract thousands of loose files onto physical disk!
• Atomic Scoop-and-Wipe (--scoop): When packaging projects, the chunker reconciles the SHA-256 hash of every single archived file against the source before pruning the source directory. Byte-level integrity guaranteed.

Subsystem G: PathGuard Defense & Transaction State

When autonomous agents or CLI tools dynamically resolve paths, malicious actors love to inject path traversal attacks (like ../../../../etc/shadow or symlink escapes). The Core family defends against this via bejson_path_guard:

• Strict Sandbox Boundaries: Validates all file targets against an approved root sandbox anchor.
• Symlink Resolution: Traces canonical symlink targets. If a symlink points outside the sandbox boundary, the operation is blocked and raises Code 24 (E_PATH_TRAVERSAL_DETECTED)!

Meanwhile, bejson_state_management provides an in-memory commit journal. If an agent performs a 5-step mutation and fails on step 4, bejson_state_rollback() restores the document to its pre-transaction state without requiring a disk re-read.

Chapter 8: Subsystems H & I — The 4-Language API Parity Matrix & Token Economics

We’ve covered every layer of the architecture: low-level memory, atomic disk persistence, O(1) field maps, 20-point validation, MFDB federation, in-cell nesting, and security sandboxing. Now, let’s bring it all together into the definitive 4-Language API Reference Matrix and analyze the Token Economics that make BEJSON 104 the undefeated champion for AI-native architectures.

The Universal 4-Language API Parity Matrix

Every single core operational primitive is mapped 1:1 across Python, JavaScript, TypeScript, and Bash:

Subsystem H: The Canonical Self-Tracking Ledger

The 104 Core family practices what it preaches. Under libraries_104/BEJSON/, the entire library ecosystem tracks its own metadata using an internal MFDB database:

• 104a.mfdb.bejson: Master ecosystem manifest.
• data/families.bejson: Definitive registry of all 35 library families across Python, JS, TS, and Bash.
• data/errors.bejson: Master catalog of all 490+ global error codes and diagnostic tokens.
• data/dependencies.bejson: Directed acyclic graph mapping 200+ cross-module import relationships.

The AI Token Economics: Why Traditional Databases Fail Local Agents

Why do autonomous AI agents (like Antigravity and Claude) perform 10x better on BEJSON than on standard JSON or SQL? The numbers don't lie:

Because the schema is declared exactly once in Fields, schema overhead approaches 0% as rows grow. Autoregressive LLMs never suffer from attention drift or hallucinated dictionary keys because the AI only generates pure positional value vectors. Furthermore, the static header block integrates natively with Prefix Prompt Caching on Gemini and Claude, cutting inference costs by over 70%!