How HKUDS/nanobot Works

User sends a message and gets a tool-using assistant response

A user starts in the CLI or any connected chat app and sends a normal request such as asking a question, searching the web, or editing files in the workspace. The gateway or CLI converts that request into a normalized inbound event, the runtime loads the matching session, reconstructs context from workspace rules, long-term memory, skills, and recent chat history, and checks whether the prompt is too large to safely fit the model. The model then receives the request together with all available tools. If it decides to call tools, the runtime executes them, may stream progress and concise tool hints back to the user, and loops until a final answer is ready. The completed assistant reply is saved into session history, old context may be archived in the background, and the final response is delivered back to the originating interface unless the turn already sent its answer through the messaging tool.

1. User sends a message through CLI or a connected chat channel
2. Gateway or CLI normalizes the request and applies sender or channel admission rules
3. Runtime loads the session, rebuilds context, and checks memory pressure
4. Model runs with the current tool catalog and may request actions
5. Runtime executes tools, emits progress updates when applicable, and iterates until completion
6. Final answer is saved into session history and delivered back to the original surface

Operator deploys and runs nanobot across local and chat surfaces

An operator first initializes the product, which creates a config file, workspace, and default templates while preserving existing values when possible. They then configure a model provider, choose one or more channels, and set explicit sender allowlists so the gateway can start safely. When the runtime launches, it derives media, cron, and log directories from the chosen config location, which makes multi-instance deployment predictable. If the operator prefers local use, the CLI provides a direct interactive chat surface with history and progress feedback. If they enable chat channels, the gateway discovers built-in and plugin adapters, starts only the enabled ones, and routes all inbound and outbound traffic through the shared bus. The result is one assistant runtime that can be used locally, exposed to multiple teams, or split into isolated instances per channel or workspace.

1. Operator runs onboarding to create or refresh config and workspace files
2. Operator configures a provider and chooses explicit or automatic routing
3. Operator enables channels and sets sender allowlists and group policies
4. System derives runtime data directories from the active config and optionally overrides workspace
5. Operator launches either local CLI mode or multi-channel gateway mode

Long conversations are compressed without losing durable context

As a conversation grows over many turns, the runtime estimates the true prompt size using the current provider, model, message history, and tool definitions instead of relying on a simplistic message count. If the prompt is still within the configured context window, nothing changes. Once the estimated size reaches the model limit, the system selects an older chunk of the session at a safe user-turn boundary and asks the model to convert that chunk into two assets: a searchable archive entry and an updated long-term memory state. Successful summaries advance the archived boundary so future prompts stay shorter, while the durable memory file continues to ground later responses. If summarization fails repeatedly, the system stops blocking on model quality and writes a raw archive instead. The user keeps conversation continuity, and the active prompt remains usable even in long-running sessions.

1. Runtime estimates prompt size before the next model turn
2. System selects a safe archival boundary in older session history
3. Older context is summarized into long-term memory and searchable history
4. If summarization keeps failing, system stores a raw archive instead of dropping history
5. Future turns replay only the unconsolidated recent slice plus durable memory

User delegates work to a background worker and receives the result later

During a normal chat, a user can ask for a task that is too long or too multi-step for a synchronous reply. The assistant responds by launching a background worker and immediately acknowledging that the work has started. That worker runs with a narrower tool set so it can perform local work or web tasks without recursively sending messages or spawning more workers. When the task finishes, the result is not sent directly to the user; instead, it is reinjected into the main runtime as a system-originated message linked to the original chat. The main runtime then produces a short user-facing summary and delivers it back to the same destination. This flow preserves responsiveness in the main chat while keeping the final result inside the normal conversation history.

1. User asks for a longer task during a normal conversation
2. Assistant launches a background worker and immediately confirms task creation
3. Background worker executes with a reduced tool set
4. Result is reinjected into the main runtime as a system-originated message
5. Main runtime summarizes and delivers the completion message back to the original chat

User creates reminders and proactive workspace tasks

A user can ask the assistant to remind them later, create a recurring task, or maintain a standing task list in the workspace. For explicit reminders, the assistant creates a scheduled job tied to the current chat destination so future delivery returns to the same place. The scheduler persists jobs locally, reloads them after restarts, and wakes itself when the next job is due. Separately, the heartbeat service periodically reads the workspace task file and asks the model whether there is actionable work to run right now. Even when work is executed, the system adds a second evaluation step before notifying the user, which helps suppress noisy autonomous messages. Background execution still reuses the standard agent loop and session history, so reminders and proactive actions remain part of the same conversational thread rather than becoming a disconnected notification system.

1. User asks for a reminder or recurring task during chat
2. Scheduler validates the timing mode and stores the job with the originating chat destination
3. Scheduler wakes on due time and executes the job through callbacks
4. Heartbeat periodically reviews the workspace task file and decides whether to run work
5. Autonomous results are evaluated before notification and then routed through normal conversation history

User connects and uses the assistant through WhatsApp

A user who prefers WhatsApp starts by linking the assistant through a local bridge rather than a hosted business API. If the bridge has no saved authentication state, it generates a QR code that the user scans from the WhatsApp mobile app to authorize the session. Once connected, inbound WhatsApp text and media messages are received by the local bridge, filtered to ignore self-generated traffic, and converted into a normalized payload for the Python runtime. Media files are downloaded to local storage and referenced into the agent input so the assistant can reason over them. The message then enters the same shared bus and agent loop used by other channels. Text replies are routed back through the bridge to WhatsApp, giving the user a familiar messaging surface while keeping the rest of the assistant runtime unchanged.

1. User scans a QR code to link the WhatsApp session
2. Bridge persists local auth state and reports connected status
3. Inbound WhatsApp messages and media are downloaded and normalized into local paths
4. Message enters the standard agent loop and gets processed like any other channel request
5. Final text response is returned to WhatsApp through the local bridge

Conversation runtime and tool orchestration

This module is the product's execution core: it turns each inbound message into a bounded, tool-using agent turn. The design strategy is to keep one universal conversation contract across CLI, chat channels, automation, and background work, so every request goes through the same context assembly, model invocation, tool execution, and session persistence path. The system deliberately separates user-visible progress updates from final delivery, and limits iteration depth to prevent runaway loops. The trade-off is that the runtime is simple and consistent, but throughput is constrained because processing is effectively serialized per instance.

• Unified agent turn execution — 【User Value】Users get one conversational entry point that can either answer directly or take actions with tools, without switching products or manually orchestrating steps. 【Design Strategy】Treat every incoming request as a standard agent turn with a repeatable loop: build context, ask the model, execute requested tools, feed results back, and stop only when a user-ready answer is produced or a safety boundary is hit. 【Business Logic】Step 1: When a message arrives, the runtime identifies the session using the channel and chat destination, unless an override is explicitly provided. Step 2: It loads or creates the session and reconstructs the current prompt using system instructions, long-term memory, skills, and recent conversation history. Step 3: It sends the assembled messages plus all available tool definitions to the selected model backend. Step 4: If the model returns tool requests, the runtime stores that assistant turn, executes the requested tools one by one, appends each tool result, and asks the model again. Step 5: This loop continues until the model returns normal assistant content, a provider error occurs, or the iteration ceiling is reached. Step 6: The main runtime allows at most 40 iterations in one turn. If that limit is hit, the user receives a bounded failure response asking for a smaller task. Step 7: If the provider returns an error-type finish state, the runtime sends a safe fallback message instead of storing raw provider failure text in the session. 【Trade-off】Sequential tool execution keeps behavior predictable and easy to debug, but it slows down turns that require multiple independent tool calls.
• Tool surface for files, shell, web, messaging, scheduling, delegation, and external servers — 【User Value】Users can ask the assistant to act on the local workspace, browse the web, send messages, create schedules, or extend itself through external tool servers, instead of receiving text-only answers. 【Design Strategy】Expose all actionable capabilities through one normalized tool registry, so the model sees a single catalog of functions regardless of whether the underlying capability is built-in or external. 【Business Logic】Step 1: At startup, the runtime registers a default tool set covering file reading and writing, directory listing, shell command execution, web search, web page fetching, outbound message sending, background task spawning, and scheduled task management when the scheduler is available. Step 2: If external tool servers are configured, their tools are discovered and wrapped into the same catalog with namespaced names. Step 3: Before execution, tool inputs are validated against each tool's schema, including required fields and basic bounds. Step 4: If validation fails, the runtime returns an actionable tool error rather than crashing the turn. Step 5: File operations can be limited to the workspace directory. Step 6: Shell execution can also be constrained by workspace rules and blocks clearly dangerous patterns such as destructive disk commands, recursive deletion, shutdown commands, path escape attempts, and internal network URLs. Step 7: Tool outputs are fed back into the model so the next reasoning step is based on observed results rather than assumptions. 【Trade-off】A wide tool surface makes the assistant practically useful, but safety depends heavily on configuration and the allowed tools for the current deployment.
• Progress updates during long-running turns — 【User Value】Users are less likely to think the assistant has frozen during multi-step work. 【Design Strategy】Only expose progress signals that are operationally useful, such as intermediate non-sensitive assistant text and concise tool hints, while suppressing hidden reasoning content. 【Business Logic】Step 1: Whenever the model requests tools, the runtime checks whether the assistant text before those tool calls contains user-safe progress text. Step 2: It removes any hidden thinking blocks from that text. Step 3: If any visible progress text remains, it is emitted as a progress message. Step 4: The runtime also generates a compact tool hint summarizing which tool is about to run and the first argument value, truncated to about 40 characters. Step 5: These progress events are published separately from the final answer and tagged so channels can decide whether to show them. Step 6: Channels may suppress ordinary progress, tool hints, or both based on configuration. 【Trade-off】This design improves perceived responsiveness, but users only see progress when tool calls occur; purely text-only model thinking remains silent.
• Background task delegation and reinjection — 【User Value】Users can hand off longer tasks without blocking the main conversation, while still receiving a result back in the original chat. 【Design Strategy】Create a reduced-capability background worker for long-running work, then route its result back into the normal conversation path so the final answer still feels like part of the same assistant experience. 【Business Logic】Step 1: During a normal turn, the assistant can invoke a background delegation tool. Step 2: The system immediately returns a confirmation to the user and starts a background task with a short task identifier and human-readable label. Step 3: The background worker receives a narrower tool set than the main runtime. It can use local work tools and web tools, but it cannot directly message the user or recursively delegate more tasks. Step 4: The background worker runs its own smaller agent loop, capped at 15 iterations. Step 5: When it finishes or fails, the system packages the original task plus the raw outcome and injects that package back into the main runtime as a system-originated message tied to the original chat destination. Step 6: The main runtime then performs one final summarizing turn and sends a natural-language completion message back to the user. 【Trade-off】This keeps the main conversation responsive, but background jobs are only tracked in memory and can be lost if the process restarts.
• Session commands for reset, stop, help, and restart — 【User Value】Operators and users can recover from bad states, stop runaway work, or reset context without touching files manually. 【Design Strategy】Reserve a small set of slash commands at the runtime layer so these controls work consistently across interfaces. 【Business Logic】Step 1: When a message exactly matches a reserved command, the runtime handles it before any model call. Step 2: The help command returns a fixed list of available control commands. Step 3: The new-session command archives unconsolidated history from the current chat, clears the active session, saves the new empty state, and confirms that a fresh conversation has started. Step 4: The stop command finds all active foreground tasks linked to the current session, cancels them, then also cancels any linked background workers, and reports how many tasks were stopped. Step 5: The restart command first sends a visible restarting notice, waits briefly, and then relaunches the process with the original runtime arguments. 【Trade-off】These controls are operationally useful, but within the scoped implementation they are not protected by user-level authorization checks.

Context, memory, and skill grounding

This module gives the assistant continuity and workspace awareness. Its design strategy is layered grounding: every turn is built from identity, workspace rules, long-term memory, recent unconsolidated chat history, and reusable skills. The memory system uses prompt-size-aware archival so the assistant stays usable on finite-context models without asking users to manually trim conversations. The trade-off is a practical but lightweight file-based design, which is easy to inspect and portable, but less robust for multi-user or high-scale deployments.

• Durable session replay across restarts and channels — 【User Value】Users can resume conversations after restarts and continue work in the same chat without repeating prior context. 【Design Strategy】Store each chat session as an append-only local history file keyed by chat destination, while replaying only the safe, recent, still-relevant portion back to the model. 【Business Logic】Step 1: Each session is identified by a string derived from the channel and chat destination. Step 2: On a new turn, the runtime first checks an in-memory session cache. Step 3: If no cached session exists, it loads the workspace-local session file. Step 4: If the workspace-local file is missing, it also checks a legacy global location and migrates that older file into the workspace when found. Step 5: If nothing exists, it creates a new empty session. Step 6: Every new message stored in the session receives a timestamp. Step 7: When preparing history for the model, the system starts from the last archived boundary rather than replaying the full raw transcript. Step 8: It then trims to the newest message window, with a default ceiling of 500 messages. Step 9: If the retained slice begins with assistant or tool output rather than a user turn, it drops leading non-user content until the slice starts cleanly. Step 10: It also removes leading orphan tool results when their related assistant tool request is no longer inside the retained window, so providers receive structurally legal history. 【Trade-off】This preserves compatibility and continuity, but the underlying session files keep growing because archival advances the replay boundary without compacting the stored transcript.
• Layered prompt construction from workspace rules, long-term memory, and skills — 【User Value】The assistant behaves consistently for a given workspace, remembers durable facts, and knows what reusable capabilities exist without requiring the user to restate them every turn. 【Design Strategy】Build the system prompt in ordered layers, separating stable workspace configuration from live user input. 【Business Logic】Step 1: Every turn starts with a system identity block that includes runtime environment details, workspace path, memory file locations, and skill directory locations. Step 2: The system then loads any bootstrap files that exist from a fixed allowlist: agent instructions, persona guidance, user profile, and tool policy files. Step 3: It appends only the current long-term memory file, not the historical archive file, so the prompt stays concise. Step 4: It injects the full bodies of always-on skills whose dependencies are satisfied. Step 5: It then adds an inventory of all discovered skills, including description, location, availability status, and any missing requirements, so the model can decide whether to read a full skill file later. Step 6: For the current user turn, it prepends a runtime metadata block containing current time and, when available, channel and chat identifiers. Step 7: This metadata is merged into the same user message rather than sent as a separate message, which avoids provider errors from consecutive messages with the same role. Step 8: If media paths are provided, only valid local image files are embedded; invalid or non-image files are skipped. 【Trade-off】This keeps prompts grounded without dumping the full skill catalog into every turn, but only image media is directly embedded in the current path.
• Token-triggered conversation consolidation — 【User Value】Long-running conversations stay usable even when model context windows are limited. 【Design Strategy】Estimate the real prompt footprint before each turn and archive older content only when the live prompt is close to the model limit. 【Business Logic】Step 1: Before a model call, the system estimates prompt size using the actual provider, model, rebuilt messages, and current tool definitions. Step 2: If the estimate is below the configured context window, no consolidation happens. Step 3: If the estimate reaches or exceeds the context limit, the system sets a target of reducing prompt size to about half of that limit. Step 4: It runs up to 5 archival rounds. Step 5: In each round, it accumulates enough removable content and then chooses a conservative boundary at the next user-turn edge, so the remaining prompt does not start mid-exchange or mid-tool sequence. Step 6: That chunk is handed to the memory archiving mechanism. Step 7: On success, the session's archived boundary is advanced and the prompt size is re-estimated. Step 8: A per-session lock prevents two overlapping turns from archiving the same region at the same time. 【Trade-off】This is a practical auto-shrinking policy that protects usability, but it favors conservative boundaries and does not reduce disk usage in the raw session file.
• Two-layer archival memory with raw-fallback durability — 【User Value】Important facts can persist across long conversations, while older context remains searchable even if summarization fails. 【Design Strategy】Split memory into two files: one concise current-memory file for future prompts, and one append-only history file for archival traceability. 【Business Logic】Step 1: When a chunk of old conversation is selected for archival, the system formats it with timestamps and any recorded tool activity. Step 2: It asks the model to produce a structured memory save action containing two required outputs: a grep-friendly history summary and a full replacement for the current long-term memory file. Step 3: The system first tries to force that memory-save action. Step 4: If the provider does not support forced tool choice, it retries in automatic mode. Step 5: A successful response appends the new summary to the history archive and replaces the long-term memory file only when the returned memory text differs from the existing one. Step 6: Missing tool calls, malformed arguments, missing required fields, null values, empty history summaries, or runtime exceptions all count as failures. Step 7: After 3 consecutive failures, the system stops waiting for perfect summarization and writes the original formatted conversation chunk directly into the history archive with a raw marker, then resets the failure counter. 【Trade-off】This guarantees archival durability even during model incompatibility or failure, but the long-term memory file is fully replaced rather than merged, so a bad model output can rewrite durable memory in unintended ways.
• Skill discovery with availability gating — 【User Value】Users and developers can extend the assistant with reusable capabilities while still seeing which skills are unavailable and why. 【Design Strategy】Treat skills as file-based packages discovered from both the workspace and built-in catalog, with dependency checks surfaced to the model rather than hidden. 【Business Logic】Step 1: The system scans two roots for skills: the workspace skill directory first, then the built-in skill directory. Step 2: When the same skill name exists in both places, the workspace version overrides the built-in one. Step 3: A folder counts as a skill only if it contains the required skill definition file. Step 4: For each skill, the system reads lightweight frontmatter metadata and extracts dependency requirements such as required command-line tools and required environment variables. Step 5: Skills that satisfy their requirements are marked available; unavailable ones remain visible with missing requirements listed. Step 6: Skills marked as always-on are injected directly into the prompt only if they are available. Step 7: All discovered skills are summarized in the prompt inventory so the model can selectively load a full skill definition when needed. 【Trade-off】This supports progressive disclosure and low-friction extensibility, but metadata parsing is intentionally shallow and may confuse third-party authors who expect richer YAML support.
• Custom skill scaffolding and packaging — 【User Value】Advanced users can create and share reusable skills with a predictable structure instead of inventing their own packaging conventions. 【Design Strategy】Provide built-in creator tooling that standardizes naming, validates package structure, and produces shareable archives. 【Business Logic】Step 1: Skill initialization normalizes the skill name into lowercase hyphenated format and enforces a 64-character limit. Step 2: It can optionally create standard subdirectories for scripts, references, and assets. Step 3: Validation checks that the skill contains the required definition file, includes required metadata such as name and description, uses only allowed top-level keys, and keeps the folder name aligned with the declared skill name. Step 4: Validation also checks that any always-on flag is a real boolean and that no unexpected top-level files are present beyond the allowed packaging structure. Step 5: Packaging runs validation first. Step 6: It creates a distributable archive while excluding junk directories, rejecting symbolic links, ensuring no packaged file escapes the skill root, and avoiding self-inclusion when the output archive is placed inside the source tree. 【Trade-off】This makes filesystem-based distribution safer and more consistent, but it is still local archive packaging rather than a signed or centrally trusted marketplace model.

Channel gateway and local operator surfaces

This module turns the assistant into a multi-surface product instead of a single runtime. The design strategy is a decoupled gateway: channels normalize inbound events into a shared bus, while outbound messages are routed back through platform-specific adapters. In parallel, the CLI acts as a first-class local control plane for setup, testing, and direct usage. The product value is broad reach with one agent core, while the main trade-off is uneven platform behavior because each channel still owns its own formatting, media handling, and retry details.

• Multi-channel gateway with built-in and plugin discovery — 【User Value】Operators can expose the same assistant in multiple chat products without rebuilding the agent for each platform. 【Design Strategy】Use a shared channel contract and registry-driven discovery so built-in channels and third-party plugins are started the same way. 【Business Logic】Step 1: On gateway startup, the system discovers all built-in channel adapters and any installed plugin adapters. Step 2: It checks configuration for each discovered channel and instantiates only the ones marked enabled. Step 3: Each enabled channel starts its own long-running listener task. Step 4: A separate outbound dispatcher starts in parallel and listens for agent replies from the shared message bus. Step 5: Channel adapters are responsible for platform-specific connection logic, but all of them publish normalized inbound events and consume normalized outbound events through the same shared runtime contract. 【Trade-off】This architecture makes transport extensibility straightforward, but discovery alone does not standardize platform behavior; each adapter still has to implement reliable send and receive logic itself.
• Fail-closed sender access control and group routing policy — 【User Value】Operators can keep the assistant private by default and avoid unauthorized users consuming model capacity. 【Design Strategy】Block access at the channel edge before messages enter the agent runtime, with additional group-routing policies per platform. 【Business Logic】Step 1: Every channel maintains an explicit sender allowlist. Step 2: An empty allowlist means deny all. Step 3: A wildcard entry means allow everyone. Step 4: Otherwise the sender must exactly match one of the approved identities. Step 5: During startup, the gateway validates that every enabled channel has a non-empty allowlist configuration; if any enabled channel still has an empty list, startup fails fast rather than running in a confusing deny-all state. Step 6: Some platforms add extra rules on top of sender allowlists. For example, group chats may require a bot mention, may allow open participation, or may be restricted to explicitly listed rooms. Step 7: Email adds an additional consent gate before mailbox access is allowed. 【Trade-off】This design is secure by default, but it is operationally brittle: a small configuration mistake can block all users or stop the gateway from starting.
• Outbound delivery filtering for progress, tool hints, and final replies — 【User Value】Users receive useful updates during work, but operators can suppress noisy traffic when a channel experience should stay minimal. 【Design Strategy】Tag outbound events with metadata and let the gateway decide whether to forward them before handing off to platform-specific send logic. 【Business Logic】Step 1: The outbound dispatcher continuously polls the shared outbound queue. Step 2: If an outbound message is marked as progress, the dispatcher checks whether ordinary progress messages are enabled. Step 3: If that progress message is specifically a tool hint, it separately checks whether tool hints are enabled. Step 4: Any disallowed progress event is dropped before channel delivery. Step 5: For approved messages, the dispatcher routes the content to the named channel adapter. Step 6: The adapter then reformats the message according to platform rules such as maximum length, threading, or markdown support. Step 7: Unknown channel names are logged rather than retried centrally. 【Trade-off】Central filtering keeps the gateway simple, but the actual user experience still varies significantly by platform because message rendering and delivery guarantees are adapter-specific.
• Media intake and reply-context preservation — 【User Value】Users can send images, audio, and files as part of a conversation, and the assistant can preserve reply context when the platform supports threads or quoting. 【Design Strategy】Normalize inbound media into local files and carry reply metadata through the bus so the core runtime can stay platform-agnostic. 【Business Logic】Step 1: When a channel receives attachments, it often downloads them into a local media directory before queueing the message. Step 2: Audio-capable channels may send downloaded audio through the shared transcription hook when a transcription key is available. Step 3: The channel packages any local media paths, plus metadata such as message identifiers, parent identifiers, thread roots, room identifiers, or group identifiers. Step 4: The agent runtime receives these as normalized message inputs rather than platform-native payloads. Step 5: When replying, the channel adapter uses the preserved metadata to send in-thread, quote the original message, or reply in the correct room when the platform supports that behavior. Step 6: For outbound media, the adapter enforces its own size, type, and upload constraints and may emit a visible attachment-failed message if upload is not possible. 【Trade-off】The core runtime stays clean, but attachment quality and threading behavior vary sharply across platforms.
• Interactive local CLI workspace — 【User Value】Developers and operators can use the assistant locally without any external chat platform and still get a polished, persistent chat experience. 【Design Strategy】Make the terminal a first-class channel with persistent history, markdown rendering, progress display, and safe terminal-state handling. 【Business Logic】Step 1: In interactive mode, the terminal reads multiline user input with persistent local history. Step 2: Messages are sent into the same shared bus used by external channels. Step 3: While the assistant is working, a visible loading indicator runs. Step 4: If progress or tool hints arrive, the spinner pauses so intermediate output can be printed cleanly, then resumes. Step 5: Final assistant replies are rendered with markdown support when enabled. Step 6: The terminal also aggressively flushes unread keypresses during generation to prevent stale input from leaking into the next prompt. Step 7: Interrupt and termination signals are handled so the terminal can be restored as cleanly as possible. 【Trade-off】This creates a strong local operator experience, but some terminal recovery behavior remains platform-dependent.
• Safe onboarding and multi-instance local setup — 【User Value】New users can get a working assistant quickly, while advanced operators can run multiple isolated assistant instances on the same machine. 【Design Strategy】Separate configuration path, workspace path, and runtime data directories so each instance can remain predictable and isolated. 【Business Logic】Step 1: During onboarding, the system initializes a config file and workspace structure. Step 2: If configuration already exists, it offers a choice between overwriting and preserving existing values while merging in newly introduced defaults. Step 3: Plugin discovery is run during onboarding so newly available channel schemas can be added to configuration automatically. Step 4: Workspace templates such as the heartbeat task file are synchronized into the workspace. Step 5: For normal runtime, operators can override the config path and optionally the workspace path. Step 6: Runtime directories such as media, cron, and logs are derived from the selected config file location, while the workspace comes from configuration unless explicitly overridden. Step 7: This allows separate instances, such as a Telegram bot and a Discord bot, to run simultaneously with isolated data and different ports. 【Trade-off】The model is simple and operator-friendly, but correctness depends on carefully chosen per-instance paths and ports.
• WhatsApp bridge login and inbound media handling — 【User Value】Users can talk to the assistant through WhatsApp without relying on paid cloud APIs or complex business app approval flows. 【Design Strategy】Split WhatsApp support into a local Node-based bridge for network connectivity and a Python channel adapter for integration with the core agent runtime. 【Business Logic】Step 1: The local bridge starts a WhatsApp Web client session. Step 2: If no valid authentication state exists, it generates a QR code for the user to scan with the WhatsApp mobile app. Step 3: Once authenticated, the bridge saves local session state and reports a connected status back to the Python side. Step 4: For inbound messages, the bridge filters out self-sent messages and broadcast statuses. Step 5: If a message contains media such as an image, document, or video, the bridge downloads that media to a local directory and sends the local file path plus any text caption to the Python channel. Step 6: The Python side converts those local media paths into agent-readable text markers and publishes the message into the shared bus. Step 7: To avoid duplicate processing during reconnects, the Python side keeps a recent-message cache of the last 1000 message identifiers. 【Trade-off】This provides a private and low-cost WhatsApp path, but outbound media sending is not supported and inbound media storage grows without an automatic cleanup policy.

Provider routing, model compatibility, and deployment safety

This module makes the assistant deployable across many model backends and safer to run in local or semi-trusted environments. The design strategy is a central provider registry plus common request hygiene and retry behavior, so most runtime logic stays provider-agnostic. It also adds path derivation and network validation helpers to support local multi-instance deployment and reduce obvious unsafe outbound requests. The trade-off is broad compatibility through lightweight heuristics, which is flexible but somewhat sensitive to registry order and integration consistency.

• Automatic provider matching with explicit override support — 【User Value】Operators can switch among commercial APIs, gateways, local models, and OAuth-backed providers with minimal configuration changes. 【Design Strategy】Use one registry of provider metadata to drive matching, default endpoint resolution, and provider-specific behavior instead of scattering provider checks throughout the runtime. 【Business Logic】Step 1: If the configured default provider is explicitly set to a named provider rather than automatic mode, that provider is used directly. Step 2: In automatic mode, the system first checks whether the requested model name contains an explicit provider prefix before the slash. Step 3: If that explicit prefix corresponds to an eligible provider, it wins. Step 4: If no explicit prefix wins, the system searches the ordered registry for provider keywords that match the model name. Step 5: Matching providers must still satisfy activation rules such as having credentials configured, being a local provider with a base URL, or being an explicitly matched OAuth provider. Step 6: If keyword matching fails, the system tries configured local providers with base URLs. Step 7: As a last resort, it falls back to any configured non-OAuth provider with an API key, again in registry order. Step 8: Once a provider is selected, the endpoint base URL comes from either explicit config or, for gateway and local-style providers only, a registry default. 【Trade-off】This drastically simplifies setup, but auto-selection is sensitive to registry order, so operators who want deterministic behavior should pin the provider explicitly.
• Provider-specific request shaping and compatibility handling — 【User Value】The assistant can use a wide range of model vendors without asking the user to learn each provider's quirks. 【Design Strategy】Centralize message cleanup, parameter normalization, and provider-specific payload differences before each request leaves the system. 【Business Logic】Step 1: Before sending a request, the system fills in unset generation defaults such as maximum tokens, temperature, and reasoning effort from provider configuration. Step 2: It sanitizes messages so empty content does not trigger preventable provider-side validation failures. Step 3: For gateway-style providers, model names may be prefixed, stripped, or rewritten according to registry rules. Step 4: Tool-calling requests default to automatic tool choice unless a caller explicitly asks otherwise. Step 5: Providers that support prompt caching may receive cache-control hints in the system prompt and tool definitions. Step 6: Some providers receive provider-safe shortened tool-call identifiers to avoid downstream restrictions. Step 7: Direct Azure requests use deployment-specific URLs, dedicated headers, and always send maximum completion tokens, while omitting temperature for newer reasoning-style deployment families or when reasoning effort is requested. Step 8: Direct OpenAI-compatible custom endpoints are addressed through a generic compatible client against the configured base URL. 【Trade-off】This hides most provider quirks from end users, but correctness depends on provider metadata being kept current as vendors evolve.
• Retry, degradation, and error normalization — 【User Value】Users experience fewer dead-end failures from temporary provider outages or feature mismatches. 【Design Strategy】Never let raw transport exceptions leak directly into the agent loop. Convert them into structured responses, retry when the failure looks transient, and degrade gracefully when a capability is unsupported. 【Business Logic】Step 1: Every provider call is wrapped in a safe execution layer that converts unexpected exceptions into a structured error response instead of raising them upward. Step 2: If the resulting error text looks transient, based on markers such as 429, 5xx, timeout, overloaded, connection failure, or temporarily unavailable, the request is retried after 1 second, then 2 seconds, then 4 seconds, followed by one final attempt. Step 3: If the error suggests that image input is unsupported, image content is replaced with plain text placeholders and the request is retried once without images. Step 4: Tool-call argument payloads are repaired when possible before parsing so malformed provider output does not immediately break the turn. Step 5: Final provider errors are passed back to the agent loop in normalized form so the runtime can return a user-safe fallback. 【Trade-off】This meaningfully improves reliability, but transient-failure detection is string-based rather than strongly typed, so some edge cases will still slip through.
• Local speech transcription hook — 【User Value】Users can send voice messages through supported channels and still participate in a text-based agent workflow. 【Design Strategy】Use a simple shared transcription hook rather than building a dedicated speech stack inside every channel. 【Business Logic】Step 1: Channels that support audio can hand a downloaded audio file to the shared transcription provider. Step 2: The transcription provider checks that an API key exists and that the local audio file exists. Step 3: It sends the file to a speech-to-text endpoint using a fixed speech recognition model. Step 4: If the call succeeds, it returns the transcribed text. Step 5: If the key is missing, the file is missing, or the request fails, it logs the problem and returns an empty string. 【Trade-off】This makes voice support easy to bolt onto channels, but failures degrade silently to empty text, which can make voice workflows feel unreliable.
• Basic network target validation for tool-triggered URLs — 【User Value】Operators get a first layer of protection against the assistant being tricked into probing localhost or private network targets. 【Design Strategy】Validate URLs before networked tools or shell commands act on them, focusing on obvious internal-address patterns. 【Business Logic】Step 1: When a literal URL is inspected, the validator only accepts the HTTP and HTTPS schemes. Step 2: It requires a hostname and resolves that hostname to concrete addresses. Step 3: If any resolved address falls inside blocked ranges such as loopback, private ranges, link-local, carrier-grade ranges, or local-only IPv6 ranges, the URL is rejected. Step 4: Redirect targets can be validated again after resolution. Step 5: A separate command-string scanner looks for embedded web addresses inside shell commands and flags the command if any matched URL fails validation. 【Trade-off】This is a useful SSRF-style guardrail, but it is only a helper layer; protection depends on networked tools actually calling these validators consistently.

Automation and proactive background services

This module extends the assistant from reactive chat into lightweight automation. The design strategy is to keep reminders, scheduled jobs, and proactive wake-ups inside the same conversation model instead of introducing a separate automation product. Scheduled work returns to the originating chat, and heartbeat tasks use a two-stage decision process to reduce noisy proactive messages. The trade-off is convenience over enterprise robustness: persistence is local and single-process, and some behavior depends on model compliance with tool-calling contracts.

• Conversational reminder and recurring task creation — 【User Value】Users can create reminders and recurring tasks directly in chat, without switching to a separate scheduler UI. 【Design Strategy】Expose scheduling as a tool the assistant can call during normal conversation, while binding each job to the current conversation destination. 【Business Logic】Step 1: The scheduling tool is available only when the runtime has an active scheduler and has injected the current channel and chat destination into the tool context. Step 2: To create a job, a message is mandatory. Step 3: The schedule must use one of three timing modes in practice: recurring interval, calendar-style cron expression, or one-time timestamp. Step 4: Interval schedules are stored as millisecond intervals. Step 5: One-time schedules accept an ISO-formatted timestamp and are marked for deletion after execution. Step 6: Calendar schedules accept an optional timezone, but timezone is rejected unless a cron expression is used. Step 7: Invalid timezone names or malformed one-time timestamps return explicit errors instead of creating the job. Step 8: Successful jobs are stored with delivery enabled and with the originating channel and chat destination, so future execution routes back to the same conversation. 【Trade-off】This makes scheduling feel native inside chat, but job ownership and authorization are tied only to chat context in the scoped design.
• Persistent local cron execution with restart survival — 【User Value】Scheduled jobs survive process restarts and continue firing without requiring a separate external scheduler. 【Design Strategy】Persist jobs to a local store, recompute next-run time at startup, and maintain one timer for the earliest due job. 【Business Logic】Step 1: The scheduler loads all jobs from a local JSON store on startup. Step 2: It reconstructs each job's schedule and state, recomputes the next eligible execution time for enabled jobs, saves the refreshed store, and arms the nearest timer. Step 3: Whenever the timer fires, the scheduler reloads the job store, finds all enabled jobs whose next-run timestamp is now due, and executes them one by one through a callback. Step 4: After each execution, it records the last-run time, success or error status, and the last error if one occurred. Step 5: It then updates the next-run timestamp. Step 6: One-time jobs marked for deletion are removed completely after running. Step 7: Other one-shot jobs are simply disabled after their run. Step 8: The scheduler re-arms itself to the next earliest enabled job. 【Trade-off】This achieves restart survival with very little infrastructure, but the design is single-process and file-based, so it is not safe for distributed or shared-writer deployment.
• Runaway self-scheduling guard — 【User Value】Operators are less likely to face accidental automation storms caused by one scheduled job recursively creating more jobs. 【Design Strategy】Mark executions that are already running inside a scheduled context and prohibit new job creation from that context. 【Business Logic】Step 1: Before a scheduled job is executed, the runtime can mark that turn as cron-originated. Step 2: If, during that turn, the assistant tries to create a new scheduled job, the scheduling tool checks the cron-origin flag. Step 3: If the flag is active and the requested action is job creation, the tool returns an explicit error saying new jobs cannot be scheduled from within a scheduled job execution. Step 4: Listing and removing existing jobs remain allowed even inside that scheduled context. 【Trade-off】This is a focused protection against recursive amplification, but it only works if all cron-triggered execution paths correctly set the cron-origin flag.
• Heartbeat-driven proactive task review — 【User Value】Users can maintain a standing task list in the workspace and let the assistant periodically decide whether anything needs to be acted on. 【Design Strategy】Use the model itself to interpret open-ended heartbeat instructions, but force that decision into a simple structured action of skip or run. 【Business Logic】Step 1: At a fixed interval, defaulting to every 30 minutes, the heartbeat service wakes up if enabled. Step 2: It reads the task file from the workspace. Step 3: If the file is missing or empty, it exits quietly. Step 4: If content exists, it sends the current time plus the task file content to the model and exposes a single heartbeat decision tool. Step 5: The model is expected to return either a skip action or a run action, optionally with a natural-language summary of the tasks to execute. Step 6: If no tool call is returned, the heartbeat defaults to skip. Step 7: Only the run branch triggers downstream execution. 【Trade-off】This avoids fragile rule parsing and allows natural-language automation instructions, but correctness depends on the model actually following the heartbeat tool-call contract.
• Two-stage notification filter for proactive output — 【User Value】Users are less likely to be spammed by low-value autonomous messages. 【Design Strategy】Separate the decision to execute a heartbeat task from the decision to notify the user about its output. 【Business Logic】Step 1: If heartbeat decides to run, the service delegates actual task execution through a callback. Step 2: If execution produces no response, nothing is delivered. Step 3: If execution produces a response, that response is passed through a second evaluation step together with the original task summary. Step 4: Delivery happens only when that evaluator explicitly approves the output and a notification callback is configured. Step 5: If evaluation rejects the response, the result is intentionally silenced. 【Trade-off】This adds a useful anti-noise filter, but the actual quality threshold lives outside the core heartbeat module.
• Automation routed through normal session history — 【User Value】Background work, scheduled jobs, and proactive assistant actions remain part of the same ongoing conversation rather than appearing as disconnected system events. 【Design Strategy】Reuse the normal agent loop and session persistence path for automation-triggered work instead of building a parallel execution engine. 【Business Logic】Step 1: For direct automated execution, the runtime can synthesize a normal inbound message and send it through the standard processing path. Step 2: For system-originated background messages, the runtime decodes the origin channel and chat destination from a channel-and-chat encoding convention. Step 3: It loads the corresponding session, rebuilds context from that session's history plus the new system content, and runs the same model-and-tool loop used for ordinary user turns. Step 4: New assistant and tool messages are persisted into the same session history, with oversized tool outputs still truncated at 16,000 characters. Step 5: The resulting assistant output is delivered back to the decoded destination as a normal outbound message. 【Trade-off】This keeps automation consistent with ordinary conversations, but it relies on a string-encoding convention for origin routing rather than a more explicit typed routing object.

A full agent runtime in a tiny operational footprint

Problem: How do you deliver a real tool-using assistant, with memory, channels, automation, and background work, without inheriting the complexity and maintenance cost of a large agent framework?

Solution: Step 1: The product centers everything on one reusable agent-turn contract rather than separate runtimes for CLI, channels, cron, and background tasks. Step 2: All surfaces publish normalized inbound events into the same bus and receive normalized outbound events back. Step 3: Context building, provider invocation, tool execution, session persistence, and memory consolidation all happen in one common path. Step 4: Optional capabilities such as MCP tools, cron, heartbeat, and subagents are bolted onto that same path instead of branching into separate subsystems. The smart part is that product breadth comes from composition rather than architectural sprawl. That is how the project can support many surfaces and workflows while staying unusually small and understandable.

Technologies: Python asyncio, shared message bus, tool-calling LLM pattern, file-based persistence

Boundaries & Risks: This simplicity is strongest in single-user or operator-owned deployments. It becomes less ideal when high concurrency, strict tenancy isolation, or deep enterprise observability are required, because the compact architecture also means fewer built-in coordination and governance layers.

Prompt-size-aware memory compression with guaranteed archival fallback

Problem: How do you keep long conversations usable on limited-context models without forcing users to manually trim history or losing old context when summarization fails?

Solution: Step 1: Before a turn, the system estimates the full prompt footprint using the real provider, model, rebuilt history, and current tool definitions. Step 2: If the prompt is too large, it selects an older chunk at a conservative user-turn boundary so the live context remains structurally coherent. Step 3: It asks the model to produce two outputs at once: a concise searchable history entry and a full updated long-term memory state. Step 4: It retries with a softer tool-choice mode when forced structured output is unsupported. Step 5: If repeated summarization attempts fail, it still writes a raw archive after 3 consecutive failures. The cleverness is the durability guarantee: semantic compression is preferred, but persistence never depends on the model behaving perfectly.

Technologies: token estimation, LLM function calling, asyncio locks, Markdown memory files

Boundaries & Risks: Stored session files are not compacted, so disk usage still grows. The durable memory file is replaced wholesale rather than merged, so a poor model update can overwrite long-term memory in unintended ways.

One bus-driven assistant across many chat products and local CLI

Problem: How do you expose the same assistant in many interfaces without hard-wiring platform logic into the core agent?

Solution: Step 1: Every interface implements the same channel contract for start, stop, send, and inbound normalization. Step 2: All inbound traffic is converted into a common event shape with channel, sender, chat, content, media, and metadata. Step 3: The agent core processes these events without knowing platform details. Step 4: Outbound replies are filtered for progress semantics centrally, then delegated back to each adapter for platform-specific rendering, threading, or attachment handling. Step 5: New adapters can be discovered as plugins rather than changing the core runtime. The smart part is that transport diversity is contained at the edges while the core keeps one conversation model.

Technologies: asyncio queues, plugin discovery via entry points, channel adapter contract, normalized inbound and outbound events

Boundaries & Risks: Channel behavior is uneven because reliability, formatting, media support, and retry logic still live inside each adapter. Product consistency therefore depends on ongoing adapter maintenance.

Registry-driven provider portability across direct, gateway, local, and OAuth model backends

Problem: How do you support many model vendors without scattering provider-specific branching across the entire application?

Solution: Step 1: Provider metadata is centralized in a single registry that describes naming keywords, endpoint behavior, gateway rules, direct-call behavior, and capability flags. Step 2: Runtime selection uses this metadata to auto-match or explicitly pin a provider. Step 3: Request shaping then applies provider-specific rules such as model prefixing, endpoint selection, payload adjustments, prompt caching hints, and reasoning parameter handling. Step 4: Shared retry and message sanitation logic sits above the transport layer so most failures are normalized before the agent loop sees them. The cleverness is that adding broad backend coverage mostly becomes a metadata exercise rather than a rewrite of core runtime logic.

Technologies: provider registry, Pydantic config schema, LiteLLM, OpenAI-compatible clients, httpx

Boundaries & Risks: Auto-selection depends on registry order, so behavior can shift if providers are reordered or newly added. Some integrations also rely on process-level environment mutation, which is less suitable for embedded or multi-tenant hosting.

Local-first extensibility through skills and external tool servers

Problem: How do you let users extend assistant behavior without modifying the core codebase or rebuilding the runtime?

Solution: Step 1: Skills are discovered from both the workspace and the built-in catalog, with workspace versions taking precedence. Step 2: Each skill advertises metadata about availability and dependencies so the assistant can see what exists without blindly loading everything. Step 3: Always-on skills are injected automatically, while all other skills are summarized for on-demand loading. Step 4: In parallel, external MCP servers can be connected and their tools exposed as native agent tools, whether they run locally or remotely. The smart part is progressive disclosure: the assistant gets a compact capability map by default and expands only when necessary, keeping prompts smaller while preserving extensibility.

Technologies: Markdown skill packages, dependency-aware skill discovery, MCP, workspace overrides

Boundaries & Risks: Skill metadata parsing is intentionally shallow, which may limit third-party packaging complexity. External tool servers also bring their own latency, auth, and availability risks.

Autonomous work routed back into the same conversation model

Problem: How do you add reminders, heartbeat checks, and delegated background work without creating a second product experience separate from chat?

Solution: Step 1: Scheduled jobs and heartbeat tasks are not delivered through a separate notification engine. Step 2: Instead, they are converted into direct or system-originated messages that re-enter the normal agent loop. Step 3: That loop uses the same session, tools, memory, and delivery logic as ordinary user messages. Step 4: Heartbeat adds a second evaluation gate before notifying the user, which helps reduce noise from proactive runs. The cleverness is consistency: reactive chat and proactive automation share one conversation history and one execution model, which keeps user experience coherent and implementation compact.

Technologies: cron-style local scheduler, callback-based services, system-message reinjection, session persistence

Boundaries & Risks: The scheduler is single-process and file-based, so it is not suitable for distributed execution. Heartbeat behavior also depends on model compliance with structured tool-calling.

Private low-cost WhatsApp access without a hosted cloud API

Problem: How do you offer WhatsApp access for a personal assistant without requiring a business cloud integration, public webhook infrastructure, or recurring API costs?

Solution: Step 1: A local Node bridge manages the WhatsApp Web session, QR-based device linking, reconnect behavior, and media downloading. Step 2: That bridge binds only to the local machine and can require a bridge token handshake, so it is not exposed as a public service. Step 3: The Python side connects to the bridge over WebSocket and converts bridge events into standard assistant messages. Step 4: Downloaded media is passed as local file paths into the agent runtime so multimodal context can be handled without changing the core loop. The smart part is cost and privacy: the system piggybacks on a local linked-device session rather than external cloud messaging infrastructure.

Technologies: Node.js WebSocket bridge, Baileys WhatsApp Web client, localhost binding, Python WebSocket channel adapter

Boundaries & Risks: This approach is suitable for personal or operator-run setups, not for formal enterprise messaging products that require stronger compliance guarantees. Outbound media sending is also not implemented, and local media storage currently has no cleanup policy.

Practical local safety rails for tools and network access

Problem: How do you make a local tool-using assistant safer by default without building a heavy policy engine?

Solution: Step 1: File tools can be restricted to the configured workspace so path escape attempts are rejected. Step 2: Shell execution checks for clearly dangerous command patterns such as destructive operations, shutdown commands, and path traversal. Step 3: Network validators inspect literal URLs and URLs embedded in shell commands, allowing only normal web schemes and rejecting hostnames that resolve into loopback, private, link-local, carrier-grade, or local IPv6 ranges. Step 4: These helpers can also validate redirect targets after resolution. The cleverness is not perfect security, but high leverage: a small number of concrete checks meaningfully reduces the most obvious local risks in operator-owned deployments.

Technologies: workspace path restriction, command-pattern guards, URL target validation, private-network blocking

Boundaries & Risks: These are guardrails rather than full sandboxing. Enforcement is only as strong as the tools that actually call these validators, and the scoped evidence does not prove end-to-end enforcement everywhere.