New Agent: Technical Architecture and User Guide
By Leethaxor69
Table of Contents
- Chapter 1: Section 1: Introduction to New Agent
- Chapter 2: Section 2: Technical Architecture Overview
- Chapter 3: Section 3: Environment Setup and Dependencies
- Chapter 4: Section 4: The Flask Web Terminal Interface
- Chapter 5: Section 5: Job Creation and Task Management
- Chapter 6: Section 6: Amnesia and Memory Rebirth Protocols
- Chapter 7: Section 7: Configuration and Key Management
- Chapter 8: Section 8: Troubleshooting and Error Handling
- Chapter 9: Section 9: Security Policy and Best Practices
Chapter 1: Section 1: Introduction to New Agent
Overview of New Agent
Listen up, noobs. If you’re trying to wrap your head around New Agent, stop thinking of it as just another "AI chat tool." It’s a full-stack, Flask-based terminal environment designed to give you direct, programmatic control over agentic workflows. Whether you're running it via the CLI or the web terminal, it relies on the same core do_exec() engine that keeps the wheels turning.
New Agent was built for people who actually want to get things done, not for script kiddies looking for a toy. It doesn't rely on bloated, inefficient overhead; it uses BEJSON (Boehnen Elton JSON) (Boehnen Elton JSON) to maintain rigorous positional integrity for your data. This isn't just "some JSON"—it’s a structured, self-validating data standard that keeps your configuration, history, and jobs from becoming a corrupted mess.
Core Philosophy
The design of New Agent revolves around three critical mandates:
- Atomic Integrity: We don't do partial writes. Every critical operation—like memory compression, job state updates, or configuration changes—is executed atomically. If the system crashes, you don't end up with a half-baked, broken file.
- Structural Blindness: We’ve separated administrative concerns from operational context. By using
Network_Roleheaders (Mastervs.Slave), the system ensures that your operational nodes (the Slaves) stay lean, focused, and free of unnecessary administrative bloat, while the Master node maintains the authoritative registry. - Persistence over Preamble: We don't waste context windows on "hello, how are you" chatter. New Agent is built to consume and output data in a way that respects your compute resources. It uses direct, task-oriented action tags (like
<job_task_done/>) to ensure the model behaves like a tool, not a therapist.
The New Agent Ecosystem
You aren't just installing a script; you're deploying a tiered, federated architecture. Here is the high-level breakdown of what you’re dealing with:
- The Engine: The underlying logic shared across all entry points (
agent.py,cliagent.py,webagent.py). It handles the heavy lifting of action execution, memory management, and BEJSON parsing. - The Web Terminal: A Flask-based interface that isn't just a UI—it’s a functional terminal wrapper. It features a custom tab system, an integrated job manager, and a note-taking module, all while mirroring the exact logic of the CLI.
- The Data Layer: Fully compliant with the BEJSON spec v1.31. Whether you are dealing with
104(entity files),104a(metadata/configs), or theMFDB (Multi-File Database)(multi-file database) architecture, New Agent validates everything. If your file isn't structurally sound, the system fails closed.
If you're still confused, read the docs. The lib/ directory is full of the source logic you'll need to understand if you plan on modifying the system. Don't go breaking things until you understand how the Parent_Hierarchy resolution works—if you mess with the pathing, your database becomes an orphan, and that’s on you.
Get your environment set up properly, source your files correctly using newagent_source_env(), and stop hardcoding paths like an amateur. We’re building tools, not prototypes.
Chapter 2: Section 2: Technical Architecture Overview
System Architecture: The Tri-Layer Logic
If you want to understand how New Agent actually functions, stop treating it like a "black box" AI. It is a strictly architected, multi-layered system designed for high-throughput task execution and local state persistence. The technical architecture relies on three distinct layers that handle the separation of concerns: the Orchestration Layer, the Execution Engine, and the State Persistence Layer.
1. The Orchestration Layer (Flask/TUI)
The web interface isn't just a pretty face for the browser—it's a full Flask-based terminal wrapper (webagent.py). Unlike amateur implementations that rely on WebSocket overhead or bloated frameworks, this layer interfaces directly with the do_exec() engine.
- Zero-Redundancy Command Path: The UI and the CLI entry points call the exact same
do_exec()subprocess engine. Whether you click "Start" in the browser or type a command in the terminal, the execution path remains identical. - Structural Blindness Protocol: As defined in the v1.31 MFDB spec, the interface respects the
Network_Roleheader. If the node is marked as aSlave, it remains "blind" to the Master’s global registry, keeping the local context window lean and minimizing token waste on administrative metadata.
2. The Execution Engine (The "Core")
The core logic resides in the lib/ directory. This is where the heavy lifting happens, specifically for the AI's action-execution loop.
- Action Atomicity: When the system processes an action—like
<amnesia_rebirth/>or<job_task_done/>—it uses an atomic file-swap strategy (viaos.rename). This prevents partial writes. If the process is killed mid-action, your state on disk remains pristine, not corrupted. - Bounded Loop Protection: We implement a "circuit breaker" logic for the AI's internal loops. With a
MAX_AUTO_CONTINUEconstant, the agent is forced to yield control back to the environment. It doesn't run indefinitely; it executes, checks state, and awaits confirmation.
3. The State Persistence Layer (BEJSON v1.31)
This is where 90% of you will fail if you don't pay attention. New Agent uses BEJSON as its data foundation, not standard JSON.
- Positional Integrity: Because we use
BEJSON 104for entities and104afor configuration/manifests, the data is stored in a fixed-matrix format. Every field is mapped to an index. If you try to hack in a new column without updating theFieldsheader, the validator will nuke your session. - Hierarchical Resolution: Every entity file (e.g.,
data/jobs.bejson) must contain aParent_Hierarchykey pointing back to the manifest. If you move these files around like a standard file system, you’ll break the back-reference, the validator will mark the database as "Orphaned," and the system will stop processing your data.
_fk suffix for foreign keys and the manifest's primary_key registry. If you are performing joins outside of these conventions, you are bypassing the architectural safety controls and introducing "silent failure" points. Use the provided lib_bejson_validator.py; if it returns an error, fix the structure before you complain about the system being broken.
Data Lifecycle & Federation
The lifecycle of a job—from creation in the UI to execution in the Engine—follows a strictly unidirectional flow:
- Registry Load: Upon
newagent_source_env(), the system hydratesos.environwith the correct paths. - Manifest Verification: The engine reads the
104a.mfdb.bejsonmanifest to verify theentity_nameandfile_pathof available entity stores. - Entity Ingestion: Data is loaded into the
ContextBubble. If the model requests an action, the state is updated, and theValuesarray is rewritten atomically. - Log Distillation: The operational logs are stripped and pushed upwards (in Master/Slave setups), ensuring the active
Context/directory never exceeds its threshold for meaningful input.
If you are planning to extend the architecture, keep your new modules within the lib/ directory and ensure they satisfy the BEJSON positional requirements. If you deviate from these architectural mandates, you aren't "modifying" the system—you're creating technical debt that will eventually break your persistence layer.
Chapter 3: Section 3: Environment Setup and Dependencies
Listen up, noobs. If you’re trying to run the New Agent without a clean environment, you’re going to get hit with a cascade of ImportError exceptions that will waste everyone's time. This project isn't some bloated, dependency-heavy web framework garbage; it’s lean, but it relies on specific, strictly controlled inputs. If your sys.path isn't configured exactly as the architecture expects, the engine will fail to resolve the library chain, and your entire persistence layer will go dark.
1. Minimal Dependency Requirements
Don't go installing the entire PyPI registry. The core engine is built to be modular, but you need the foundational network and request handling libraries. Your environment must include:
aiohttp: Required for the asynchronous event loop and handling the non-blocking network requests the agent uses to interface with models.requests: Used for the synchronous blocking calls and internal API interaction between thewebagent.pyFlask wrapper and the execution engine.- Flask: Necessary to host the web terminal interface.
If you aren't using a virtual environment (venv) to isolate these, you're asking for dependency hell. Use pip install aiohttp requests flask and keep it inside a dedicated directory.
2. The lib/ Path Injection
The engine expects all structural logic, validators, and BEJSON core utilities to reside in the lib/ directory. If the script can't find the lib_bejson_newagent_* modules, the newagent_source_env() call will fail immediately.
In your implementation, ensure you are resolving the path dynamically at the top of your entry point (like webagent.py does) so you don't break the environment when you move the root directory:
import sys
from pathlib import Path
# Add the lib directory to sys.path before importing core modules
sys.path.insert(0, str(Path(__file__).resolve().parent / "lib"))
# Now the engine can find its dependencies
import lib_bejson_newagent_env as newagent_env
If you skip this, your PYTHONPATH will be misaligned, and the system won't be able to hydrate the os.environ variables required for storage resolution.
3. Environment Sourcing Protocol
Stop hardcoding paths. The architecture uses a tiered sourcing strategy (lib_bejson_newagent_env.newagent_source_env()) to populate INTERNAL_STORAGE and SD_CARD paths.
- The Secure/Paths Chain: The system looks for
secure.pyandpaths.pyfirst. These are where you define your environment variables. - Legacy Fallback: It only falls back to
env_file.pyif the newer modular files are missing. - Validation: If
get_env_path()returnsNoneor a broken directory string, your BEJSON persistence layer will crash upon the first write attempt. Always verify your environment variables are populated by running a quick debug print ofos.environbefore launching the agent.
4. Database Directory Hierarchy
The engine requires a specific layout. If your directory structure doesn't match the manifest expectations, the MFDB validator will flag the database as "broken" or "orphaned" before the model even initializes.
Ensure your root directory matches this structure:
/config/- Storesconfig.json,keys.bejson, andmodels.bejson./lib/- Contains the structural modules (do not touch these unless you want to break core features)./logs/- Operational logs./Context/- The state of the current session; this must be writable by the user running the agent./data/- The primary entity store (if using MFDB orchestration).
Pro-tip: If you’re manually creating these, ensure permissions are set correctly. The engine performs atomic file swaps (os.rename). If the process doesn't have write/execute permissions on the directory, the atomic swap will fail, and you’ll end up with corrupted BEJSON state files. Don't come crying to the logs when you lose your session data because you were lazy with your chmod settings.
Chapter 4: Section 4: The Flask Web Terminal Interface
Stop trying to re-invent the wheel. webagent.py isn't some bloated web server for your pretty UI—it’s a lightweight Flask wrapper specifically designed to give you a terminal-like GUI without duplicating the logic already baked into agent.py. It uses the exact same do_exec() subprocess engine as the CLI version, so if your commands fail here, they’ll fail in the CLI too. Don't waste my time trying to debug the shell logic in the browser; debug the agent engine.
1. Functional Scope and Design
The web interface is a sibling entry point. It shares the project directory, the config/ directory, and the lib/ dependencies. If you change a key or a model in your terminal, the web interface sees it instantly. It provides the following critical UI components:
- Command Terminal: A high-fidelity, scanline-aesthetic terminal. It interacts with the
do_exec()action tag logic. - Persistent Notes: A
/api/notesendpoint that writes tonotes/webagent_notes.txt. No BEJSON overhead here—it’s just raw text because keeping a whole database record for a sticky note is idiotic. - Job Management: The Jobs tab directly manipulates
self._active_job_pathandself._active_job_doc. You start or stop jobs here, and the AI context updates on the next turn.
2. Communication and Atomicity
The browser terminal communicates with the backend via a clean REST API. If you’re building on top of this, stick to these endpoints:
- POST /api/compress: Executes
bubble.run_full_session_compression(). This is destructive. It wipes the live history and reseeds it with the recap. It uses a fail-closed protocol; if the compression process trips over its own feet, the original history remains untouched. - POST /api/amnesia: Splits the old compress-and-wipe logic. It wipes
self.history, dumps a recap toContext/amnesia_recap.txt, and optionally triggers a "reborn" state ifauto_amnesia_memory_retrievalis enabled. - Atomic Updates: Every update to the session context or configuration uses
os.renamefor atomic file swaps. If you find a half-writtenkey_state.bejson, it’s because your OS or disk I/O crashed during a swap.
3. Frontend Aesthetics and UX
The UI is built with a hard-coded CSS injection that prioritizes visibility and the "hacker" aesthetic without sacrificing usability.
- Scanlines: Global
.scanlinesoverlay with 30% opacity. Don't turn this off; it's there to keep the CRT look clean. - Brand Red: All inputs use
--brand-redfor borders, inverting to white text on red background only when focused. This is so you don't get lost in the dark terminal environment. - Tab Bar: Config is locked to the far right. We use
flex-wrap: nowrap; if your screen resolution is tiny, the tabs will scroll horizontally instead of squishing the labels into unreadable garbage.
4. Implementation Warning: The "Command" Tab
If you see a blank "Command" tab in the first position, don't touch it. It’s a placeholder. I put it there for future functionality, but if you start trying to wire it to an untested engine, you're going to break the api_chat loop. Leave the placeholder alone until you understand how the RestPrompter and ModelRegistry act as the primary communication middleware.
5. Proper Shutdown and Security
Because webagent.py pulls in the full lib_bejson_newagent_env sourcing protocol, it inherits the same security policy as the CLI. If you aren't using the newagent_source_env() function, your environment variables won't be hydrated, and the file-path resolver will return None.
If the web agent fails to start, check the logs/ directory. If you don't see a clear error, you've likely failed to inject the lib/ directory into sys.path.
# The webagent.py boot-up sequence requires these
import sys
from pathlib import Path
# Add library path explicitly before importing lib modules
sys.path.insert(0, str(Path(__file__).resolve().parent / "lib"))
# Now verify the environment
import lib_bejson_newagent_env as newagent_env
newagent_env.newagent_source_env()
If you try to run this outside of the project root, it won't resolve the relative paths to Context/ or logs/. This isn't a portable binary; it's a structural engine. Treat it as such.
Chapter 5: Section 5: Job Creation and Task Management
Listen up, noob. If you’re trying to track work without using the dedicated Jobs tab, you’re doing it wrong. The Job Creation system is hard-wired to keep the AI focused. We don’t run arbitrary shell loops; we define a task, assign a goal, and track the progress. The system is located in the jobs/ directory, and it handles everything via the self._active_job_path and self._active_job_doc state objects.
1. Architectural Flow
The Jobs system is decoupled from the chat loop but injected into the build_system_prompt() function every single request. Here is why that matters: if you change a job’s status via the web interface, the model isn't "guessing" what it should be doing—it's reading the active job context directly.
- Job Initialization: When you start a job from the UI, the backend creates a doc in the
jobs/directory. - Completion Tracking: Once a job hits its final state, it’s moved to
jobs/complete/. - The AI Interface: The AI only ever sees the state of the currently active job. If you aren't running a job, that context field is null. Don't try to force the AI to manage a queue; it’s an execution engine, not a project manager.
2. Manual Task Control
If you’re not using the browser GUI, you’re forced to trigger jobstart or manual completion tags. The system is designed to be atomic. If the agent hits a <job_task_done/> tag, it clears the current task, updates the progress in the BEJSON document, and—if that was the final task—it triggers the job completion move.
3. State Management in the Web UI
The Jobs tab is just a view layer. When you click "Start" or "Stop" in the UI, you aren't running a command; you are performing a POST request to /api/jobs/start or /api/jobs/stop. This hits the lib_bejson_newagent_jobs module to flip the active flag in the document.
4. Avoiding "Zombie" Jobs
One of the most common mistakes I see noobs make is leaving zombie jobs in the jobs/ directory. If you have a job document in jobs/ that isn't actually being worked on, it might confuse the agent during the prompt construction phase.
- Verify: Always ensure that if a job is in
jobs/, it has anactiveflag set totrue. - Cleanup: If a job failed and you don't intend to resume, move it to
jobs/complete/or delete it entirely. - No AI Intervention: Stop trying to ask the model "what jobs do I have?". The model is not an indexer. If you want to see what's pending, look at the directory yourself or check the Jobs tab in the web terminal. The system is built for execution, not for you to play project manager with the LLM.
5. Task Completion Protocols
Every task has a specific identifier. When the agent completes a step, it must emit the <job_task_done/> tag. If it doesn't, the backend has no way to increment the task progress counter in your BEJSON document. If you're writing custom actions, remember to include the job_task_id attribute in your completion responses, or the system won't know which task specifically was finished.
If you screw up the ID, the system will error out on the next turn. Keep your IDs clean, keep your state locked, and stop overcomplicating it. It’s a simple state machine; treat it like one.
Chapter 6: Section 6: Amnesia and Memory Rebirth Protocols
Listen up. If you’re letting your agent’s history file bloat into a gigabyte-scale mess of dead logs and irrelevant context, you're just begging for hallucination loops. The New Agent isn't designed to be a historical archive; it’s an execution engine. When the context window gets noisy, you wipe it. That’s what the Amnesia and Memory Rebirth protocols are for.
Don't panic—this isn't a factory reset. It's a high-level compression and distillation process.
1. The Mechanics of Amnesia
When you trigger the /api/amnesia endpoint (via the header button in the web terminal), you are initiating a destructive, yet controlled, compression of the agent's current history object.
- The Wipe: The system unconditionally clears the live model-facing memory.
- The Recap: Before the wipe is finalized,
bubble.save_amnesia_recap()runs. This generates a condensed, human-readable summary of the agent’s accomplishments, active state, and pending goals. - Persistence: This recap is saved to
Context/amnesia_recap.txt.
This action is fail-closed. If the compression engine hits a snag, the history remains untouched. You don’t lose your data; the logger on disk remains entirely pristine and independent of the live compression state.
2. Automatic vs. Manual Rebirth
You have two ways to handle the "rebirth" phase, determined by your auto_amnesia_memory_retrieval config setting.
- Auto-Rebirth (
True): The system immediately feeds theamnesia_recap.txtback into the agent's context window. It’s a seamless continuity loop. The agent "wakes up" knowing its recent history without the baggage of the raw, bloated log. - Manual Rebirth (
False): You get a true blank slate. The agent knows nothing. This is for when you want to shift the agent to an entirely different objective and don't want old context influencing the new workflow. To bring it back, you have to manually triggerPOST /api/rebirth, which executesbubble.load_amnesia_recap().
3. The Compression Pipeline
The backend uses bubble.run_full_session_compression() to perform the heavy lifting. This isn't just a str.replace operation—it’s a structured distillation. It maps the active bubble state into a core truth set.
4. Integrity and Logging
Stop worrying about your logs. The Amnesia protocol is strictly isolated to the active chat memory. Your on-disk transcript logs are never compressed, deleted, or truncated during this process. If you need to debug what the agent did before it was "amnesiac," the files in logs/ are your source of truth.
5. Common Noob Traps
- Triggering Amnesia during a Task: Don't do it. If you trigger amnesia while an
active_jobis in progress, you are creating a context disconnect. The agent might "forget" the specificjob_task_idit was tracking. Always wait for a task completion or a clean break before wiping the buffer. - Ignoring the Recap: If you’re manually handling the rebirth, read the
Context/amnesia_recap.txtfile first. If the recap is garbage, the reborn agent will be hallucinating trash. Garbage in, garbage out. - Over-Compression: There is no reason to run Amnesia every 10 minutes. It's a resource-heavy state-summarization task. Run it when you reach a major milestone, not when you’re mid-sentence.
Keep the context clean, keep the goals explicit, and use the rebirth protocol to prevent the agent from wandering off into the weeds of a stale conversation. If it’s not relevant to the current objective, it doesn't belong in the window.
Chapter 7: Section 7: Configuration and Key Management
Listen up, noobs. You want your agent to actually do something besides crash? Then stop hardcoding your API keys like a complete amateur. New Agent uses a tiered configuration system that separates your sensitive credentials from your operational logic. If I catch you committing an api_key string directly into a .py file, don't come crying to me when your accounts get pwned.
1. The Configuration Hierarchy
New Agent relies on a strict environment-sourcing protocol. Before the engine even initializes, lib_bejson_newagent_env.newagent_source_env() is called. This function scans your environment for secure storage paths.
- Primary Source:
config/config.json. This is where your operational settings live. Keep yourauto_amnesia_memory_retrievalsettings and other UI defaults here. - Key Storage:
config/keys.bejson. This is where the magic happens. We use a BEJSON 104 structure for this because we need absolute positional integrity. If your keys aren't structured, your keys aren't secure. - The Fallback: If you’re too lazy to set up the proper BEJSON directory structure, the system falls back to a legacy
env_file.pychain. Don’t use it. It’s a security liability.
2. Managing the keys.bejson Vault
Your keys.bejson file isn't just a list; it’s a database of your identities. By using the BEJSON 104 format, we ensure that every key is tied to a specific type and provider.
When the agent starts, it calls rest.sync_keys_from_env_sources(). This function parses your keys.bejson and populates the runtime environment. If the file is malformed, the agent fails closed. That’s a feature, not a bug—if you can't manage your config, you shouldn't have access to the agent.
3. Atomic Updates via os.rename
You want to rotate a key? Don't edit the file while the agent is running.
The system implements an atomic file swap. You create your new keys.bejson in a temp directory and use os.rename to overwrite the active configuration. The engine polls the directory for these changes. By using atomic operations, we prevent the "partial-read" scenario where the agent tries to load a half-written file and nukes its own connection state.
4. Best Practices for the Paranoid
If you're not encrypting your keys, you're basically handing them out on a silver platter.
- Encryption at Rest: Ensure your
config/directory is chmod 600. If anyone else on your system can read these files, they deserve to pwn you. - Environment Variables: Use
get_env_path()to reference your storage locations. Never use hardcoded absolute paths likeC:\Users\Noob\Desktop\keys.json. That’s how you guarantee your code breaks the moment you move to a production server or a clean container. - Key Rotation: Rotate your keys monthly. If you’re using the same API key from 2024, you’re an easy target. The
sync_keys_from_env_sourcesutility is designed to handle live refreshes; use it.
5. Common Noob Traps
- The "Permission Denied" Panic: If your agent fails to start, it's usually because your
config/directory doesn't have the right permissions. Check youros.environsettings. IfINTERNAL_STORAGEisn't pointing to a writable directory, the agent can't create the necessary state files. - Mixing Formats: Don't put your config in a 104db file. Configs are 104a or 104. 104db is for relational data—stop trying to jam your API keys into a relational schema that doesn't belong there.
- Ignoring the Validator: We have
lib_bejson_validator.jsfor a reason. If you edit yourkeys.bejsonmanually and break the structure, the validator will catch it. If you force the agent to run despite validation errors, you are asking for undefined behavior.
Keep your keys in the vault, keep your config structure strictly compliant with BEJSON 104, and stop treating the config/ directory like a dumping ground for junk files. If you can't keep your environment clean, you can't keep your agent secure. Period.
Chapter 8: Section 8: Troubleshooting and Error Handling
Listen up, because I’m only going to explain this once. If your agent is failing, it’s 99% likely because you didn't read the documentation or you broke the structural integrity of a BEJSON file. Stop guessing and start debugging like you have a clue.
If you see an error, the first thing you do is check logs/. If the log is empty, your environment isn't even initialized. If you’re getting "Invalid Format" errors, you’ve messed up the positional integrity of your Values arrays.
1. The "Fail-Closed" Philosophy
New Agent follows a strict "fail-closed" policy. If a BEJSON document is corrupt, the parser will refuse to load it. Do not try to bypass this by editing the validator code. If the validator catches a failure in the Values length vs. Fields length, it means your data is corrupted. If you try to force that data into the engine, you’ll get garbage-in, garbage-out, or worse—a complete crash of the memory state.
2. Identifying Validation Errors
Most of your issues will stem from the lib_bejson_validator.js or the equivalent Python logic in lib/.
- Error Code 30–49 (MFDB): These are architectural failures. If you see code 33 (Entity file not found), check your
Parent_Hierarchypath. You probably moved the file without updating the relative link. - Positional Integrity Failures: If you added a field to your
Fieldsarray but forgot to update every row inValueswith a correspondingnull(or value), the validator will dump an error. Use a proper validator script, don't edit these files in Notepad like a caveman. - Type Mismatches: If your schema says
integerbut you passed astringinside yourValuesrow, the validator will scream. Every field must match its declared type.
3. Debugging the Flask Terminal
If the webagent.py interface is giving you "Internal Server Error" when you try to hit an endpoint (like /api/chat or /api/compress), check the server-side console output.
- Permission Denied: Check
chmod 600on yourconfig/andlogs/directories. If the process doesn't own the directory, it can't write the state files. - Memory Refusal: If you’re running
/api/compressand it fails, it’s usually because the history was too corrupted to parse for theamnesia_recap.txt. The system will refuse to clear the history if it can't verify the recap, keeping your transcript safe. Check the last entry inContext/to see what the agent was doing before it choked.
4. Resolving Orphaned States
Sometimes your key_state.bejson or models.bejson might get out of sync if you kill the process mid-write.
- Backup first: Copy the
config/directory to a safe place before you touch anything. - Verify the Manifest: If you are using MFDB, check the manifest (
104a.mfdb.bejson) to ensure thefile_pathmatches the current disk layout. - Clean the Temp: If you were in the middle of an atomic
os.renameand the system crashed, you might have a stale temporary file. Delete it. - Re-sync: Run the initialization sequence again. If the engine detects a missing state, it will try to re-index the available files based on the
Records_Type.
5. Common "Noob" Error Table
| Symptom | Cause | Fix |
|---|---|---|
TypeError: Cannot read property 'x' of null |
You tried to access a field that wasn't properly initialized in the Values row. |
Ensure null padding is applied correctly to that row. |
| Agent won't start; "Missing mandatory key" | You deleted one of the six BEJSON top-level keys. | Restore the key. Don't touch the boilerplate header. |
FileNotFound during load |
Parent_Hierarchy is pointing to a dead end. |
Update the relative path string in the entity file. |
Changes in config.json not reflecting |
You didn't restart the agent. | Configs are loaded at init. Restart the terminal. |
| UI is frozen/Scanlines only | Web terminal websocket connection dropped. | Refresh the page. If it persists, check the Flask log for socket timeout. |
If you’re still failing after checking these, you’re either missing dependencies in requirements.txt or you’re running an incompatible Python version. Check your logs, check your structure, and stop breaking the rules. If you want a GUI that works, respect the architecture.
Chapter 9: Section 9: Security Policy and Best Practices
Listen up. Security isn't a suggestion; it’s the only thing standing between your data and a total system wipe. If you’re treating this agent like a toy, you’re going to get pwned. This section covers how to lock down your NewAgent deployment so you don't end up with an exploited sandbox.
1. The "Fail-Closed" Security Model
The engine operates on a strict fail-closed policy. If the lib_bejson_validator.js or the Python lib_bejson_newagent_errors.py logic throws an exception during a document load, do not ignore it. If a document fails structural integrity, the system will prevent execution. Attempting to override the validator to "just make it work" is how you create injection vulnerabilities. If the Values length doesn't match the Fields length, the data is untrusted. Purge it.
2. File System Permissions and Hardening
The config/ directory is the crown jewel of your setup. It contains keys.bejson, key_state.bejson, and models.bejson.
- Restrict Ownership: Ensure the
config/directory is owned by the user running the agent process, and no one else. - Permissions: Use
chmod 600for all sensitive.bejsonfiles. If a file is world-readable, consider your API keys compromised. - Atomic Operations: The system relies on
os.renamefor atomic file updates. If you see orphan.tmpfiles in yourconfig/directory, the process was killed mid-write. Delete them manually, but never delete the primary files unless you have a backup.
3. Encryption at Rest
Do not store your API keys or sensitive model configurations in plaintext if you are deploying in a multi-user environment. While the BEJSON format supports plain text for ease of parsing, your environment should encrypt the config/ directory at the disk level (e.g., LUKS, dm-crypt). If you are passing these files over a network, ensure they are encrypted in transit via TLS 1.3.
4. Injection Prevention
The agent executes code through the do_exec() subprocess engine. This is a powerful, dangerous tool.
- Never expose the
webagent.pyinterface to a public-facing network without an authenticated reverse proxy (e.g., Nginx with OIDC or mutual TLS). - Sanitization: The
webagent.pyterminal interface is a wrapper. It does not sanitize command input because it expects the underlyingagent.pyengine to handle execution logic. If you are modifying the shell logic, assume every string coming from the browser is a malicious payload.
5. Memory Security (Amnesia & Rebirth)
The Amnesia protocol (/api/amnesia) is your best friend. It forces a state reset and compresses your history recap.
- Auto-Amnesia: If you enable
auto_amnesia_memory_retrieval, the system re-seeds the model with the recap. Ensureamnesia_recap.txtis not world-readable. It contains the abstracted history of your session—if a threat actor steals that file, they have the distilled summary of your agent's entire activity.
6. Summary Checklist for Secure Deployment
| Category | Action | Why? |
|---|---|---|
| Identity | Rotate keys.bejson every 30 days. |
Mitigates impact of compromised key material. |
| Filesystem | chmod 700 on logs/ and config/. |
Prevents unauthorized file access. |
| Validation | Enable strict mode in the validator. | Catches malformed packets that might trigger buffer overflows. |
| Network | Deploy behind a VPN/authenticated proxy. | Prevents remote execution via web terminal. |
| Logging | Audit logs/ for unexpected shell activity. |
Detects unauthorized command execution patterns. |
7. Handling Compromise
If you suspect the agent environment has been compromised:
- Kill the process immediately.
- Back up the entire
logs/andContext/directories for forensic analysis. - Invalidate all keys listed in
keys.bejson. - Wipe the
config/andContext/directories. - Reinitialize the environment using the source files in
lib/.
Don't be a noob—secure your environment or don't use it at all. The architecture provides the tools for safety, but if you leave the front door unlocked, the system can't protect you.