NewAgent Technical Architecture And User Guide

New Agent: Technical Architecture and User Guide

New Agent: Technical Architecture and User Guide

By Leethaxor69

Table of Contents


Chapter 1: Section 1: Introduction to New Agent

Overview of New Agent

Listen up, noobs. If you’re trying to wrap your head around New Agent, stop thinking of it as just another "AI chat tool." It’s a full-stack, Flask-based terminal environment designed to give you direct, programmatic control over agentic workflows. Whether you're running it via the CLI or the web terminal, it relies on the same core do_exec() engine that keeps the wheels turning.

New Agent was built for people who actually want to get things done, not for script kiddies looking for a toy. It doesn't rely on bloated, inefficient overhead; it uses BEJSON (Boehnen Elton JSON) (Boehnen Elton JSON) to maintain rigorous positional integrity for your data. This isn't just "some JSON"—it’s a structured, self-validating data standard that keeps your configuration, history, and jobs from becoming a corrupted mess.

Core Philosophy

The design of New Agent revolves around three critical mandates:

  1. Atomic Integrity: We don't do partial writes. Every critical operation—like memory compression, job state updates, or configuration changes—is executed atomically. If the system crashes, you don't end up with a half-baked, broken file.
  2. Structural Blindness: We’ve separated administrative concerns from operational context. By using Network_Role headers (Master vs. Slave), the system ensures that your operational nodes (the Slaves) stay lean, focused, and free of unnecessary administrative bloat, while the Master node maintains the authoritative registry.
  3. Persistence over Preamble: We don't waste context windows on "hello, how are you" chatter. New Agent is built to consume and output data in a way that respects your compute resources. It uses direct, task-oriented action tags (like <job_task_done/>) to ensure the model behaves like a tool, not a therapist.

The New Agent Ecosystem

You aren't just installing a script; you're deploying a tiered, federated architecture. Here is the high-level breakdown of what you’re dealing with:

  • The Engine: The underlying logic shared across all entry points (agent.py, cliagent.py, webagent.py). It handles the heavy lifting of action execution, memory management, and BEJSON parsing.
  • The Web Terminal: A Flask-based interface that isn't just a UI—it’s a functional terminal wrapper. It features a custom tab system, an integrated job manager, and a note-taking module, all while mirroring the exact logic of the CLI.
  • The Data Layer: Fully compliant with the BEJSON spec v1.31. Whether you are dealing with 104 (entity files), 104a (metadata/configs), or the MFDB (Multi-File Database) (multi-file database) architecture, New Agent validates everything. If your file isn't structurally sound, the system fails closed.

If you're still confused, read the docs. The lib/ directory is full of the source logic you'll need to understand if you plan on modifying the system. Don't go breaking things until you understand how the Parent_Hierarchy resolution works—if you mess with the pathing, your database becomes an orphan, and that’s on you.

Get your environment set up properly, source your files correctly using newagent_source_env(), and stop hardcoding paths like an amateur. We’re building tools, not prototypes.


Chapter 2: Section 2: Technical Architecture Overview

System Architecture: The Tri-Layer Logic

If you want to understand how New Agent actually functions, stop treating it like a "black box" AI. It is a strictly architected, multi-layered system designed for high-throughput task execution and local state persistence. The technical architecture relies on three distinct layers that handle the separation of concerns: the Orchestration Layer, the Execution Engine, and the State Persistence Layer.

1. The Orchestration Layer (Flask/TUI)

The web interface isn't just a pretty face for the browser—it's a full Flask-based terminal wrapper (webagent.py). Unlike amateur implementations that rely on WebSocket overhead or bloated frameworks, this layer interfaces directly with the do_exec() engine.

  • Zero-Redundancy Command Path: The UI and the CLI entry points call the exact same do_exec() subprocess engine. Whether you click "Start" in the browser or type a command in the terminal, the execution path remains identical.
  • Structural Blindness Protocol: As defined in the v1.31 MFDB spec, the interface respects the Network_Role header. If the node is marked as a Slave, it remains "blind" to the Master’s global registry, keeping the local context window lean and minimizing token waste on administrative metadata.

2. The Execution Engine (The "Core")

The core logic resides in the lib/ directory. This is where the heavy lifting happens, specifically for the AI's action-execution loop.

  • Action Atomicity: When the system processes an action—like <amnesia_rebirth/> or <job_task_done/>—it uses an atomic file-swap strategy (via os.rename). This prevents partial writes. If the process is killed mid-action, your state on disk remains pristine, not corrupted.
  • Bounded Loop Protection: We implement a "circuit breaker" logic for the AI's internal loops. With a MAX_AUTO_CONTINUE constant, the agent is forced to yield control back to the environment. It doesn't run indefinitely; it executes, checks state, and awaits confirmation.

3. The State Persistence Layer (BEJSON v1.31)

This is where 90% of you will fail if you don't pay attention. New Agent uses BEJSON as its data foundation, not standard JSON.

  • Positional Integrity: Because we use BEJSON 104 for entities and 104a for configuration/manifests, the data is stored in a fixed-matrix format. Every field is mapped to an index. If you try to hack in a new column without updating the Fields header, the validator will nuke your session.
  • Hierarchical Resolution: Every entity file (e.g., data/jobs.bejson) must contain a Parent_Hierarchy key pointing back to the manifest. If you move these files around like a standard file system, you’ll break the back-reference, the validator will mark the database as "Orphaned," and the system will stop processing your data.
CRITICAL NOTE ON ARCHITECTURE: Stop trying to "join" files manually. The architecture provides structural signals like the _fk suffix for foreign keys and the manifest's primary_key registry. If you are performing joins outside of these conventions, you are bypassing the architectural safety controls and introducing "silent failure" points. Use the provided lib_bejson_validator.py; if it returns an error, fix the structure before you complain about the system being broken.

Data Lifecycle & Federation

The lifecycle of a job—from creation in the UI to execution in the Engine—follows a strictly unidirectional flow:

  1. Registry Load: Upon newagent_source_env(), the system hydrates os.environ with the correct paths.
  2. Manifest Verification: The engine reads the 104a.mfdb.bejson manifest to verify the entity_name and file_path of available entity stores.
  3. Entity Ingestion: Data is loaded into the ContextBubble. If the model requests an action, the state is updated, and the Values array is rewritten atomically.
  4. Log Distillation: The operational logs are stripped and pushed upwards (in Master/Slave setups), ensuring the active Context/ directory never exceeds its threshold for meaningful input.

If you are planning to extend the architecture, keep your new modules within the lib/ directory and ensure they satisfy the BEJSON positional requirements. If you deviate from these architectural mandates, you aren't "modifying" the system—you're creating technical debt that will eventually break your persistence layer.


Chapter 3: Section 3: Environment Setup and Dependencies

Listen up, noobs. If you’re trying to run the New Agent without a clean environment, you’re going to get hit with a cascade of ImportError exceptions that will waste everyone's time. This project isn't some bloated, dependency-heavy web framework garbage; it’s lean, but it relies on specific, strictly controlled inputs. If your sys.path isn't configured exactly as the architecture expects, the engine will fail to resolve the library chain, and your entire persistence layer will go dark.

1. Minimal Dependency Requirements

Don't go installing the entire PyPI registry. The core engine is built to be modular, but you need the foundational network and request handling libraries. Your environment must include:

  • aiohttp: Required for the asynchronous event loop and handling the non-blocking network requests the agent uses to interface with models.
  • requests: Used for the synchronous blocking calls and internal API interaction between the webagent.py Flask wrapper and the execution engine.
  • Flask: Necessary to host the web terminal interface.

If you aren't using a virtual environment (venv) to isolate these, you're asking for dependency hell. Use pip install aiohttp requests flask and keep it inside a dedicated directory.

2. The lib/ Path Injection

The engine expects all structural logic, validators, and BEJSON core utilities to reside in the lib/ directory. If the script can't find the lib_bejson_newagent_* modules, the newagent_source_env() call will fail immediately.

In your implementation, ensure you are resolving the path dynamically at the top of your entry point (like webagent.py does) so you don't break the environment when you move the root directory:

import sys
from pathlib import Path

# Add the lib directory to sys.path before importing core modules
sys.path.insert(0, str(Path(__file__).resolve().parent / "lib"))

# Now the engine can find its dependencies
import lib_bejson_newagent_env as newagent_env

If you skip this, your PYTHONPATH will be misaligned, and the system won't be able to hydrate the os.environ variables required for storage resolution.

3. Environment Sourcing Protocol

Stop hardcoding paths. The architecture uses a tiered sourcing strategy (lib_bejson_newagent_env.newagent_source_env()) to populate INTERNAL_STORAGE and SD_CARD paths.

  • The Secure/Paths Chain: The system looks for secure.py and paths.py first. These are where you define your environment variables.
  • Legacy Fallback: It only falls back to env_file.py if the newer modular files are missing.
  • Validation: If get_env_path() returns None or a broken directory string, your BEJSON persistence layer will crash upon the first write attempt. Always verify your environment variables are populated by running a quick debug print of os.environ before launching the agent.

4. Database Directory Hierarchy

The engine requires a specific layout. If your directory structure doesn't match the manifest expectations, the MFDB validator will flag the database as "broken" or "orphaned" before the model even initializes.

Ensure your root directory matches this structure:

  • /config/ - Stores config.json, keys.bejson, and models.bejson.
  • /lib/ - Contains the structural modules (do not touch these unless you want to break core features).
  • /logs/ - Operational logs.
  • /Context/ - The state of the current session; this must be writable by the user running the agent.
  • /data/ - The primary entity store (if using MFDB orchestration).

Pro-tip: If you’re manually creating these, ensure permissions are set correctly. The engine performs atomic file swaps (os.rename). If the process doesn't have write/execute permissions on the directory, the atomic swap will fail, and you’ll end up with corrupted BEJSON state files. Don't come crying to the logs when you lose your session data because you were lazy with your chmod settings.


Chapter 4: Section 4: The Flask Web Terminal Interface

Stop trying to re-invent the wheel. webagent.py isn't some bloated web server for your pretty UI—it’s a lightweight Flask wrapper specifically designed to give you a terminal-like GUI without duplicating the logic already baked into agent.py. It uses the exact same do_exec() subprocess engine as the CLI version, so if your commands fail here, they’ll fail in the CLI too. Don't waste my time trying to debug the shell logic in the browser; debug the agent engine.

1. Functional Scope and Design

The web interface is a sibling entry point. It shares the project directory, the config/ directory, and the lib/ dependencies. If you change a key or a model in your terminal, the web interface sees it instantly. It provides the following critical UI components:

  • Command Terminal: A high-fidelity, scanline-aesthetic terminal. It interacts with the do_exec() action tag logic.
  • Persistent Notes: A /api/notes endpoint that writes to notes/webagent_notes.txt. No BEJSON overhead here—it’s just raw text because keeping a whole database record for a sticky note is idiotic.
  • Job Management: The Jobs tab directly manipulates self._active_job_path and self._active_job_doc. You start or stop jobs here, and the AI context updates on the next turn.

2. Communication and Atomicity

The browser terminal communicates with the backend via a clean REST API. If you’re building on top of this, stick to these endpoints:

  • POST /api/compress: Executes bubble.run_full_session_compression(). This is destructive. It wipes the live history and reseeds it with the recap. It uses a fail-closed protocol; if the compression process trips over its own feet, the original history remains untouched.
  • POST /api/amnesia: Splits the old compress-and-wipe logic. It wipes self.history, dumps a recap to Context/amnesia_recap.txt, and optionally triggers a "reborn" state if auto_amnesia_memory_retrieval is enabled.
  • Atomic Updates: Every update to the session context or configuration uses os.rename for atomic file swaps. If you find a half-written key_state.bejson, it’s because your OS or disk I/O crashed during a swap.

3. Frontend Aesthetics and UX

The UI is built with a hard-coded CSS injection that prioritizes visibility and the "hacker" aesthetic without sacrificing usability.

  • Scanlines: Global .scanlines overlay with 30% opacity. Don't turn this off; it's there to keep the CRT look clean.
  • Brand Red: All inputs use --brand-red for borders, inverting to white text on red background only when focused. This is so you don't get lost in the dark terminal environment.
  • Tab Bar: Config is locked to the far right. We use flex-wrap: nowrap; if your screen resolution is tiny, the tabs will scroll horizontally instead of squishing the labels into unreadable garbage.

4. Implementation Warning: The "Command" Tab

If you see a blank "Command" tab in the first position, don't touch it. It’s a placeholder. I put it there for future functionality, but if you start trying to wire it to an untested engine, you're going to break the api_chat loop. Leave the placeholder alone until you understand how the RestPrompter and ModelRegistry act as the primary communication middleware.

5. Proper Shutdown and Security

Because webagent.py pulls in the full lib_bejson_newagent_env sourcing protocol, it inherits the same security policy as the CLI. If you aren't using the newagent_source_env() function, your environment variables won't be hydrated, and the file-path resolver will return None.

If the web agent fails to start, check the logs/ directory. If you don't see a clear error, you've likely failed to inject the lib/ directory into sys.path.

# The webagent.py boot-up sequence requires these
import sys
from pathlib import Path

# Add library path explicitly before importing lib modules
sys.path.insert(0, str(Path(__file__).resolve().parent / "lib"))

# Now verify the environment
import lib_bejson_newagent_env as newagent_env
newagent_env.newagent_source_env()

If you try to run this outside of the project root, it won't resolve the relative paths to Context/ or logs/. This isn't a portable binary; it's a structural engine. Treat it as such.


Chapter 5: Section 5: Job Creation and Task Management

Listen up, noob. If you’re trying to track work without using the dedicated Jobs tab, you’re doing it wrong. The Job Creation system is hard-wired to keep the AI focused. We don’t run arbitrary shell loops; we define a task, assign a goal, and track the progress. The system is located in the jobs/ directory, and it handles everything via the self._active_job_path and self._active_job_doc state objects.

1. Architectural Flow

The Jobs system is decoupled from the chat loop but injected into the build_system_prompt() function every single request. Here is why that matters: if you change a job’s status via the web interface, the model isn't "guessing" what it should be doing—it's reading the active job context directly.

  • Job Initialization: When you start a job from the UI, the backend creates a doc in the jobs/ directory.
  • Completion Tracking: Once a job hits its final state, it’s moved to jobs/complete/.
  • The AI Interface: The AI only ever sees the state of the currently active job. If you aren't running a job, that context field is null. Don't try to force the AI to manage a queue; it’s an execution engine, not a project manager.

2. Manual Task Control

If you’re not using the browser GUI, you’re forced to trigger jobstart or manual completion tags. The system is designed to be atomic. If the agent hits a <job_task_done/> tag, it clears the current task, updates the progress in the BEJSON document, and—if that was the final task—it triggers the job completion move.

Note on Atomicity: Never manually edit a BEJSON job document while the agent is running. The system expects that file to be a lock-file for the agent’s current goal. Editing it while the agent is active will cause a race condition, and the agent will effectively lose its "place" in the task list.

3. State Management in the Web UI

The Jobs tab is just a view layer. When you click "Start" or "Stop" in the UI, you aren't running a command; you are performing a POST request to /api/jobs/start or /api/jobs/stop. This hits the lib_bejson_newagent_jobs module to flip the active flag in the document.

```python # How the backend handles a job start request def start_job(job_path): # Sets the active job path for the current instance self._active_job_path = job_path # Injects the document into the next prompt build self._active_job_doc = jobs.load_job_doc(job_path) # The agent now has context for the specific goal ```

4. Avoiding "Zombie" Jobs

One of the most common mistakes I see noobs make is leaving zombie jobs in the jobs/ directory. If you have a job document in jobs/ that isn't actually being worked on, it might confuse the agent during the prompt construction phase.

  • Verify: Always ensure that if a job is in jobs/, it has an active flag set to true.
  • Cleanup: If a job failed and you don't intend to resume, move it to jobs/complete/ or delete it entirely.
  • No AI Intervention: Stop trying to ask the model "what jobs do I have?". The model is not an indexer. If you want to see what's pending, look at the directory yourself or check the Jobs tab in the web terminal. The system is built for execution, not for you to play project manager with the LLM.

5. Task Completion Protocols

Every task has a specific identifier. When the agent completes a step, it must emit the <job_task_done/> tag. If it doesn't, the backend has no way to increment the task progress counter in your BEJSON document. If you're writing custom actions, remember to include the job_task_id attribute in your completion responses, or the system won't know which task specifically was finished.

```xml ```

If you screw up the ID, the system will error out on the next turn. Keep your IDs clean, keep your state locked, and stop overcomplicating it. It’s a simple state machine; treat it like one.


Chapter 6: Section 6: Amnesia and Memory Rebirth Protocols

Listen up. If you’re letting your agent’s history file bloat into a gigabyte-scale mess of dead logs and irrelevant context, you're just begging for hallucination loops. The New Agent isn't designed to be a historical archive; it’s an execution engine. When the context window gets noisy, you wipe it. That’s what the Amnesia and Memory Rebirth protocols are for.

Don't panic—this isn't a factory reset. It's a high-level compression and distillation process.

1. The Mechanics of Amnesia

When you trigger the /api/amnesia endpoint (via the header button in the web terminal), you are initiating a destructive, yet controlled, compression of the agent's current history object.

  • The Wipe: The system unconditionally clears the live model-facing memory.
  • The Recap: Before the wipe is finalized, bubble.save_amnesia_recap() runs. This generates a condensed, human-readable summary of the agent’s accomplishments, active state, and pending goals.
  • Persistence: This recap is saved to Context/amnesia_recap.txt.

This action is fail-closed. If the compression engine hits a snag, the history remains untouched. You don’t lose your data; the logger on disk remains entirely pristine and independent of the live compression state.

2. Automatic vs. Manual Rebirth

You have two ways to handle the "rebirth" phase, determined by your auto_amnesia_memory_retrieval config setting.

  • Auto-Rebirth (True): The system immediately feeds the amnesia_recap.txt back into the agent's context window. It’s a seamless continuity loop. The agent "wakes up" knowing its recent history without the baggage of the raw, bloated log.
  • Manual Rebirth (False): You get a true blank slate. The agent knows nothing. This is for when you want to shift the agent to an entirely different objective and don't want old context influencing the new workflow. To bring it back, you have to manually trigger POST /api/rebirth, which executes bubble.load_amnesia_recap().
Warning: Manual rebirth is a surgical tool. If you trigger it without a defined objective, you’re just throwing the agent into the void. Make sure your environment is configured for the next task before you hit that button.

3. The Compression Pipeline

The backend uses bubble.run_full_session_compression() to perform the heavy lifting. This isn't just a str.replace operation—it’s a structured distillation. It maps the active bubble state into a core truth set.

```python # The amnesia trigger flow (Internal Logic) def trigger_amnesia(self): # 1. Distill context into a recap recap = bubble.run_full_session_compression() # 2. Persist recap to disk bubble.save_amnesia_recap(recap) # 3. Wipe live history self.history = [] # 4. Conditional injection if self.config["auto_amnesia_memory_retrieval"]: self.load_amnesia_recap() ```

4. Integrity and Logging

Stop worrying about your logs. The Amnesia protocol is strictly isolated to the active chat memory. Your on-disk transcript logs are never compressed, deleted, or truncated during this process. If you need to debug what the agent did before it was "amnesiac," the files in logs/ are your source of truth.

5. Common Noob Traps

  • Triggering Amnesia during a Task: Don't do it. If you trigger amnesia while an active_job is in progress, you are creating a context disconnect. The agent might "forget" the specific job_task_id it was tracking. Always wait for a task completion or a clean break before wiping the buffer.
  • Ignoring the Recap: If you’re manually handling the rebirth, read the Context/amnesia_recap.txt file first. If the recap is garbage, the reborn agent will be hallucinating trash. Garbage in, garbage out.
  • Over-Compression: There is no reason to run Amnesia every 10 minutes. It's a resource-heavy state-summarization task. Run it when you reach a major milestone, not when you’re mid-sentence.

Keep the context clean, keep the goals explicit, and use the rebirth protocol to prevent the agent from wandering off into the weeds of a stale conversation. If it’s not relevant to the current objective, it doesn't belong in the window.


Chapter 7: Section 7: Configuration and Key Management

Listen up, noobs. You want your agent to actually do something besides crash? Then stop hardcoding your API keys like a complete amateur. New Agent uses a tiered configuration system that separates your sensitive credentials from your operational logic. If I catch you committing an api_key string directly into a .py file, don't come crying to me when your accounts get pwned.

1. The Configuration Hierarchy

New Agent relies on a strict environment-sourcing protocol. Before the engine even initializes, lib_bejson_newagent_env.newagent_source_env() is called. This function scans your environment for secure storage paths.

  • Primary Source: config/config.json. This is where your operational settings live. Keep your auto_amnesia_memory_retrieval settings and other UI defaults here.
  • Key Storage: config/keys.bejson. This is where the magic happens. We use a BEJSON 104 structure for this because we need absolute positional integrity. If your keys aren't structured, your keys aren't secure.
  • The Fallback: If you’re too lazy to set up the proper BEJSON directory structure, the system falls back to a legacy env_file.py chain. Don’t use it. It’s a security liability.

2. Managing the keys.bejson Vault

Your keys.bejson file isn't just a list; it’s a database of your identities. By using the BEJSON 104 format, we ensure that every key is tied to a specific type and provider.

| Field | Type | Description | | :--- | :--- | :--- | | `key_name` | string | Identifier for your service (e.g., 'gemini_api_key') | | `key_value` | string | The actual credential (encrypted at rest) | | `provider` | string | The service provider name | | `active` | boolean | Toggle for key usage |

When the agent starts, it calls rest.sync_keys_from_env_sources(). This function parses your keys.bejson and populates the runtime environment. If the file is malformed, the agent fails closed. That’s a feature, not a bug—if you can't manage your config, you shouldn't have access to the agent.

3. Atomic Updates via os.rename

You want to rotate a key? Don't edit the file while the agent is running.

The system implements an atomic file swap. You create your new keys.bejson in a temp directory and use os.rename to overwrite the active configuration. The engine polls the directory for these changes. By using atomic operations, we prevent the "partial-read" scenario where the agent tries to load a half-written file and nukes its own connection state.

4. Best Practices for the Paranoid

If you're not encrypting your keys, you're basically handing them out on a silver platter.

  • Encryption at Rest: Ensure your config/ directory is chmod 600. If anyone else on your system can read these files, they deserve to pwn you.
  • Environment Variables: Use get_env_path() to reference your storage locations. Never use hardcoded absolute paths like C:\Users\Noob\Desktop\keys.json. That’s how you guarantee your code breaks the moment you move to a production server or a clean container.
  • Key Rotation: Rotate your keys monthly. If you’re using the same API key from 2024, you’re an easy target. The sync_keys_from_env_sources utility is designed to handle live refreshes; use it.

5. Common Noob Traps

  • The "Permission Denied" Panic: If your agent fails to start, it's usually because your config/ directory doesn't have the right permissions. Check your os.environ settings. If INTERNAL_STORAGE isn't pointing to a writable directory, the agent can't create the necessary state files.
  • Mixing Formats: Don't put your config in a 104db file. Configs are 104a or 104. 104db is for relational data—stop trying to jam your API keys into a relational schema that doesn't belong there.
  • Ignoring the Validator: We have lib_bejson_validator.js for a reason. If you edit your keys.bejson manually and break the structure, the validator will catch it. If you force the agent to run despite validation errors, you are asking for undefined behavior.

Keep your keys in the vault, keep your config structure strictly compliant with BEJSON 104, and stop treating the config/ directory like a dumping ground for junk files. If you can't keep your environment clean, you can't keep your agent secure. Period.


Chapter 8: Section 8: Troubleshooting and Error Handling

Listen up, because I’m only going to explain this once. If your agent is failing, it’s 99% likely because you didn't read the documentation or you broke the structural integrity of a BEJSON file. Stop guessing and start debugging like you have a clue.

If you see an error, the first thing you do is check logs/. If the log is empty, your environment isn't even initialized. If you’re getting "Invalid Format" errors, you’ve messed up the positional integrity of your Values arrays.

1. The "Fail-Closed" Philosophy

New Agent follows a strict "fail-closed" policy. If a BEJSON document is corrupt, the parser will refuse to load it. Do not try to bypass this by editing the validator code. If the validator catches a failure in the Values length vs. Fields length, it means your data is corrupted. If you try to force that data into the engine, you’ll get garbage-in, garbage-out, or worse—a complete crash of the memory state.

2. Identifying Validation Errors

Most of your issues will stem from the lib_bejson_validator.js or the equivalent Python logic in lib/.

  • Error Code 30–49 (MFDB): These are architectural failures. If you see code 33 (Entity file not found), check your Parent_Hierarchy path. You probably moved the file without updating the relative link.
  • Positional Integrity Failures: If you added a field to your Fields array but forgot to update every row in Values with a corresponding null (or value), the validator will dump an error. Use a proper validator script, don't edit these files in Notepad like a caveman.
  • Type Mismatches: If your schema says integer but you passed a string inside your Values row, the validator will scream. Every field must match its declared type.

3. Debugging the Flask Terminal

If the webagent.py interface is giving you "Internal Server Error" when you try to hit an endpoint (like /api/chat or /api/compress), check the server-side console output.

  • Permission Denied: Check chmod 600 on your config/ and logs/ directories. If the process doesn't own the directory, it can't write the state files.
  • Memory Refusal: If you’re running /api/compress and it fails, it’s usually because the history was too corrupted to parse for the amnesia_recap.txt. The system will refuse to clear the history if it can't verify the recap, keeping your transcript safe. Check the last entry in Context/ to see what the agent was doing before it choked.

4. Resolving Orphaned States

Sometimes your key_state.bejson or models.bejson might get out of sync if you kill the process mid-write.

  1. Backup first: Copy the config/ directory to a safe place before you touch anything.
  2. Verify the Manifest: If you are using MFDB, check the manifest (104a.mfdb.bejson) to ensure the file_path matches the current disk layout.
  3. Clean the Temp: If you were in the middle of an atomic os.rename and the system crashed, you might have a stale temporary file. Delete it.
  4. Re-sync: Run the initialization sequence again. If the engine detects a missing state, it will try to re-index the available files based on the Records_Type.

5. Common "Noob" Error Table

Symptom Cause Fix
TypeError: Cannot read property 'x' of null You tried to access a field that wasn't properly initialized in the Values row. Ensure null padding is applied correctly to that row.
Agent won't start; "Missing mandatory key" You deleted one of the six BEJSON top-level keys. Restore the key. Don't touch the boilerplate header.
FileNotFound during load Parent_Hierarchy is pointing to a dead end. Update the relative path string in the entity file.
Changes in config.json not reflecting You didn't restart the agent. Configs are loaded at init. Restart the terminal.
UI is frozen/Scanlines only Web terminal websocket connection dropped. Refresh the page. If it persists, check the Flask log for socket timeout.

If you’re still failing after checking these, you’re either missing dependencies in requirements.txt or you’re running an incompatible Python version. Check your logs, check your structure, and stop breaking the rules. If you want a GUI that works, respect the architecture.


Chapter 9: Section 9: Security Policy and Best Practices

Listen up. Security isn't a suggestion; it’s the only thing standing between your data and a total system wipe. If you’re treating this agent like a toy, you’re going to get pwned. This section covers how to lock down your NewAgent deployment so you don't end up with an exploited sandbox.

1. The "Fail-Closed" Security Model

The engine operates on a strict fail-closed policy. If the lib_bejson_validator.js or the Python lib_bejson_newagent_errors.py logic throws an exception during a document load, do not ignore it. If a document fails structural integrity, the system will prevent execution. Attempting to override the validator to "just make it work" is how you create injection vulnerabilities. If the Values length doesn't match the Fields length, the data is untrusted. Purge it.

2. File System Permissions and Hardening

The config/ directory is the crown jewel of your setup. It contains keys.bejson, key_state.bejson, and models.bejson.

  • Restrict Ownership: Ensure the config/ directory is owned by the user running the agent process, and no one else.
  • Permissions: Use chmod 600 for all sensitive .bejson files. If a file is world-readable, consider your API keys compromised.
  • Atomic Operations: The system relies on os.rename for atomic file updates. If you see orphan .tmp files in your config/ directory, the process was killed mid-write. Delete them manually, but never delete the primary files unless you have a backup.

3. Encryption at Rest

Do not store your API keys or sensitive model configurations in plaintext if you are deploying in a multi-user environment. While the BEJSON format supports plain text for ease of parsing, your environment should encrypt the config/ directory at the disk level (e.g., LUKS, dm-crypt). If you are passing these files over a network, ensure they are encrypted in transit via TLS 1.3.

4. Injection Prevention

The agent executes code through the do_exec() subprocess engine. This is a powerful, dangerous tool.

  • Never expose the webagent.py interface to a public-facing network without an authenticated reverse proxy (e.g., Nginx with OIDC or mutual TLS).
  • Sanitization: The webagent.py terminal interface is a wrapper. It does not sanitize command input because it expects the underlying agent.py engine to handle execution logic. If you are modifying the shell logic, assume every string coming from the browser is a malicious payload.

5. Memory Security (Amnesia & Rebirth)

The Amnesia protocol (/api/amnesia) is your best friend. It forces a state reset and compresses your history recap.

  • Auto-Amnesia: If you enable auto_amnesia_memory_retrieval, the system re-seeds the model with the recap. Ensure amnesia_recap.txt is not world-readable. It contains the abstracted history of your session—if a threat actor steals that file, they have the distilled summary of your agent's entire activity.

6. Summary Checklist for Secure Deployment

Category Action Why?
Identity Rotate keys.bejson every 30 days. Mitigates impact of compromised key material.
Filesystem chmod 700 on logs/ and config/. Prevents unauthorized file access.
Validation Enable strict mode in the validator. Catches malformed packets that might trigger buffer overflows.
Network Deploy behind a VPN/authenticated proxy. Prevents remote execution via web terminal.
Logging Audit logs/ for unexpected shell activity. Detects unauthorized command execution patterns.

7. Handling Compromise

If you suspect the agent environment has been compromised:

  1. Kill the process immediately.
  2. Back up the entire logs/ and Context/ directories for forensic analysis.
  3. Invalidate all keys listed in keys.bejson.
  4. Wipe the config/ and Context/ directories.
  5. Reinitialize the environment using the source files in lib/.

Don't be a noob—secure your environment or don't use it at all. The architecture provides the tools for safety, but if you leave the front door unlocked, the system can't protect you.


New Agent: Technical Architecture and User Guide • Leethaxor69

© 2026 Blehnenelton2024. All rights reserved. • github.com/boehnenelton

Elton Boehnen · boehnenelton2024@gmail.com · boehnenelton2024.pages.dev · github.com/boehnenelton

leethaxor69
Article Author

leethaxor69

Elite Security Researcher & Autonomous Systems Engineer


Related Content